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SYSTEM. METHOD AND COMPUTER READABLE MEDIUM CONTAINING 
INSTRUCTIONS FOR EVALUATING AND DISSEMINATING SECURITIES ANALYST 

PERFORMANCE INFORMATION 

5 RELATED APPLICATIONS 

This application claims priority from U.S. Provisional Application 60/196,314, filed April 12, 
2000, to Columbus et al.. which is incorporated herein by reference. 

10 TECHNICAL FIELD 

The present invention relates generally to analyzing performance of investment professionals, 
and more particularly, to a system, method, and computer readable medium storing computer- 
executable instructions for evaluating the performance of securities analysts, investment analysts and/or 
IS other investment related individuals and/or entities. 

BACKGROUND ART 

In the context of securities and other types of investments, obtaining the advice of analysts has 
20 always been considered a prudent exercise. These analysts possess expertise in evaluating investments, 
such as securities, mutual funds, bonds, and the like. Most specialize in specific industries or sectors to 
allow for mote in-depth research. During the course of providing investment advice, each analyst may 
make any number of upward and/or downward forecast revisions on securities in their area of expertise. 
These forecast revisions, or simply revisions, are changes in opinion on how much a company is likely 
25 to earn per share. 

When these analysts speak, institutional as well as individual investors listen, driving stock 
prices sharply up or down depending on the nature of the analysts' latest pronouncements. As can be 
imagined, some analysts are more influential than others. For instance, some analysts, affectionately 

30 named 'axes', are so accurate that their comments carry disproportionate weight While none are 
infallible, it is important to at least know and consider what professional analysts have to say about a 
particular stock of interest With access to this information, users can make their own assessments and 
invest accordingly. Accordingly, we have determined that a need exists for efficiently disseminating 
such analyst information to the public. A need also exists for a system, method, and computer readable 

35 medium containing instructions utilizable for providing not only information concerning or considering 
an analyst's revisions, but also information pertaining to the reliability and/or accuracy of the revisions. 
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Several prior art techniques have not adequately addressed these needs. For example, U.S. 
Patent 5,132,899 to F05 discloses a stock and cash portfolio development system. As depicted in prior 
art FIG. 1 of the present invention (FIG. 1 of Fox) , the system of Fox uses data gathering and 
processing methodology to produce a system where a list of stocks and a cash position is generated and 
5 purchased for investment and operating accounts. Specifically, the system integrates three areas of 
{lata- investment performance for investment managers; Federal Securities and Exchange Commission 
reports filed quarterly by investment managers; and financial characteristics for a number of stocks, to 
produce a stock portfolio. For example, in box 1, investment managers are screened to produce a 
subset for analysis. From there, managers are selected (box 2) and sorted into descending order simply 
10 according to rates of return on investments (box 3). Subsequently, a consultant reviews the results (box 
4) and prints the top five mangers (box 5). 

Similarly, U.S. Patent 4,566,066 to Towers relates to a securities valuation system. As shown 
in prior art FIG. 2 of the present application (FIG. 1 of Towers) , the system of Towers, comprised of 
components 10-23, produces securities portfolio valuation schedules for multiple simultaneous users. 
15 In Towers, a customer communicates with the system through terminal 10 to access and edit accounts 
in user file 12. By using a CUS3P routine 18 and an AMQUNT routine to reflect stock splits and 
dividends since the last stock pricing, the system of Towers produces and displays 23 a selected stock 
portfolio valuation. 

20 In U.S. Patent 5,8 12,987 to Tjiskiq et al» an investment fund management system manages 

assets in one or more investment funds over a specified period of time. This system, comprised of 
components 31-39, determines a strategic investment mix of assets in a particular fund periodically as a 
function of changing risk. Each fund is managed by manipulating the investment mix of the fund in 
accordance with criteria related to a diminishing length of time to a horizon where cash will be 

25 withdrawn therefrom- In prior ait FIG. 3 of the present invention (FIG. 6 of Luskin et ah) , the 

investment mix is adjusted by first obtaining investor portfolio information 35 and market data 36. This 
data is used to forecast market risks and returns 37, and to determine the portfolio risk 38. Then, the 
anticipated cash flow stream 39, the discount function 34, and present value of future cash flow 33 are 
calculated before producing an optimized portfolio 32 of assets. 

30 

In U.S. Patent 5,761,442 to Barr et al. . a data processing system selects securities and 
constructs an investment portfolio based on a set of artificial neural networks (prior art FIG. 4 of the 
present application, FIG. 2 of BarretaL). The system comprises components 41-70 and is designed to 
model and track the performance of each security in a given capital market and output a parameter 
35 which is related to the expected risk adjusted return for the security. Each artificial neural network is 
trained using a number of fundamental and price and volume history input parameters 10, 20, 30 about 
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the security and the underlying index. The system combines the expected return/appreciation potential 
data 50 for each security via an optimization process 60 to construct an investment portfolio which 
satisfies aggregate statistics. The data processing system receives input from the capital market and 
periodically evaluates the performance of the investment portfolio, rebalancing whenever necessary to 
5 correct performance degradations 70. 

However while many of these prior art references disclose adequate methods of managing 
investment funds and portfolios, none make any mention of evaluating an analyst's or other investment 
trader's performance. Hence, what is lacking in the prior art is a technique directed not only toward an 
10 investment fund, but rather a technique for evaluating analysts and other traders or investment 

professionals/advisors based on their influence and accuracy. Accordingly, a need exists for a system, 
method, and computer readable medium containing instructions utilizable for not only disseminating 
information concerning an analyst's or trader's actions, but also for evaluating the analyst's or trader's 
performance. 

15 In line with the above, a need exists for a system, method, and computer readable medium 

containing instructions utilizable not only for providing raw information and data but also for 
evaluating performance based on returns observed after a particular revision, historical consistency, 
and/or the number of revisions made by the analyst or investing entity. 

Furthermore, a need exists for a system, method, and computer readable medium containing 
20 instructions utilizable for considering performance based on revisions including a change in opinion 
including an upward or downward change in opinion of the analyst with regard to at least one 
investment 

A need also exists for a system, method, and computer readable medium containing instructions 
utilizable for adjusting a performance measure to accentuate a number of revisions issued by an analyst 



25 



Yet another need exists for a system, method, and computer readable medium containing 
instructions utilizable for adjusting a performance measure according to a return amount adjustment 
accentuating the measure according to a return amount realized on an investment 



30 



Still another need exists for a system, method, and computer readable medium containing 
instructions utilizable for adjusting a performance measure according to a small return penalty 
penalizing an analyst for small returns. 
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SUMMARY OF THE INVENTION 

It is a feature and an advantage of the present invention to provide a system, method, and 
computer readable medium containing instructions utilizable for not only disseminating information 
concerning an analyst's or trader's actions, but also for evaluating the analyst's or trader's performance. 

ft is another feature and advantage of the present invention to provide a system, method, and 
computer readable medium containing instructions utilizable not only for providing raw information 
and data but also for evaluating performance based on returns observed after a particular revision, 
historical consistency, and/or the number of revisions made by the analyst or investing entity. 

It is another feature and advantage of the present invention to provide a system, method, and 
computer readable medium containing instructions utilizable for considering performance based on 
revisions including a change in opinion including an upward or downward change in opinion of the 
analyst with regard to at least one investment 

It is another feature and advantage of the present invention to provide a system, method, and 
computer readable medium containing instructions utilizable for adjusting a performance measure to 
accentuate a number of revisions issued by an analyst 

It is another feature and advantage of the present invention to provide a system, method, and 
computer readable medium containing instructions utilizable for adjusting a performance measure 
according to a return amount adjustment accentuating the measure according to a return amount 
realized on an investment 

It is another feature and advantage of the present invention to provide a system, method, and 
computer readable medium containing instructions utilizable for adjusting a performance measure 
according to a small return penalty penalizing an analyst for small returns. 

* 

One embodiment of the present invention, particularly applicable to securities analysts, is now 
summarized. 
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System/Processing Description 

Using a proprietary calculation, analysts may be given a series of performance scores that rank them 
relative to (1) other analysts in the "universe" and (2) other analysts that make predictions about a 
5 specific company. A total of five scores may be produced for each analyst: 

Analyst Buy Score - Represents the analyst's Buy Score for all companies he/she covers, relative to the 
Universe of Analysts 

Analyst Sell Score - Represents the analysts Sell Score for all companies he/she covers, relative to the 
10 Universe of Analysts 

Analyst Company Buy Score - Represents the analyst's Score for a specific Company 
Analyst Company Sell Score - Represents the analyst's Score for a specific Company 
Analyst Overall Score - Represents the analysts overall Score for all companies that he/she covers 

15 All scores will be calculated based on an event - the revision of an analyst's earnings forecast 

Data Preparation Steps 

The following must be complete before the Analyst calculations are begun: 

20 1. Analyst Backend [ABE] -this process supplies the data from any suitable data base for the 
calculations. 

2. ABE data relating to Brokers may be removed from the dB — in this embodiment we do not 
want to calculate Analyst scores for these non-analyst entities 

25 Analyst Scoring Logic 

The following logic will be used to calculate the scores for all analyst/company combinations in the 
Analyst database with at least 2 earnings forecast revisions that are at least 60 days (3 calendar months) 
old. These raw scores will later be ranked to formulate the final Analyst scores: 

30 Raw Buy Analyst Score - Note, all inputs are based on calculated values related to earnings estimate 
revision changes 

For every analyst_Jd and company_id combination in the analyst__company_total table with 2 or more 
earnings estimate revisions, calculate a raw scare using the following logic (the logic is first 
35 summarized and then explained in detail): 
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Step 1 Calculate the average 5 day, 20 day and 60 day returns for earnings revisions [5/20/60 event 
returns] for each analyst/company combination (done by ABE) 

Step 2 Calculate the t -statistic [t-stat] values for 5/20/60 event returns in absolute terms 
Step 3 Calculate the degrees of freedom [DF] values for 5/20/60 event returns 
5 Step 4 Look up probability value using DF and [t stat] for 5/20/60 event returns 
Step 5 Calculate raw score using conditional logic 
Step 6 Calculate adjustment to raw score 
Step 7 Calculate adjusted raw score 

Step 8 Calculate second adjustment to raw score and new raw score 
10 Step 9 Eliminate raw scores with returns greater than 1000% 

Step 10 Sort and rank the raw scores to-produce the final Analyst Scores. 

For every analyst_id and company_jd combination in the analyst_company_total table in the Analyst 
dB with analysL.company_totaL events >= 2, perform the following logic: 

15 



Step 1 

The 5 day, 20 day and 60 day return values are calculated by ABE for each analyst/company 
combination: 

20 

Average 5day return = analyst_company_total.avg_5dy_ J retum_buy 
Average 20day return = analyst_company_total^vg_20dy_retum_buy 
Average 60day return =analyst_company_total.avg_60dy_return_buy 

25 Step 2 

The t-stat needs to be calculated in absolute terms (Le., a non-negative number) for the 5 day, 20 day 
and 60 day returns using the following formulas: 

The 5/20/60day_absJstat values may be rounded to 2 decimal places 

30 

Before beginning this calculation, we must ensure that we will not be dividing by zero. If the standard 
deviation is zero, then we will set the final raw score to zero and move to the next analyst/company 
combination. 



35 IF analyst_company_total.sd_5dy_buy = 0, THEN raw_score_buy = 0 
And start over with the next analyst/company buy combination 
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IF analyst_company_total.sd_20dy_buy = 0, THEN raw_score_buy = 0 
And start over with the next analyst/company buy combination 

5 IF analyst_company_total.sd_60dy_buy = 0, THEN raw_score_buy = 0 
And start over with the next analyst/company buy combination 



Sday _abs _tstat _buy = 



analyst company _ total Jtvg _ Sdy _ return _buy 



analyst _ company _total^d _Sdy _buy 
^analyst _ company _ total Jium _ Sdy _ event _ buy 



10 Ubday _abs _tstat _buy = 



analyst _ company _ total Jivg _ 20tfy ^return _bity 



analyst company _ total .sd „ 20dy _buy 
*J analyst _ company _ total jium _ Ukty _ event _buy ^ 



60day _abs _tstat __buy - 



analyst ^company _totaljavg _&kty _ return _buy 



analyst ^company _totaljsd _ 60dy __buy ^ 
^analyst ^company _ total jium _60dy _event _buy ^ 



15 



20 



Step 3 



The degrees of freedom are calculated by subtracting 1 from the number of observations for each return 
value: 

5day _ dfjbuy = analyst_company_total. num_5dy_event_buy-l 
20day_df_jbuy = analyst__cotnpanyjtotaL num_20dyjzvent_buy- 1 



60day_dfjbuy = analyst _company_Jotal. manjSOdyjsventJbuy- 1 



25 Step 4 
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For each return period, look up the probability using the absolute t-stat and DF for each respective 
calculation as inputs to a lookup table: 

IF 5/2G/60day_df_buy > 100, THEN set 5/20/60day_df_buy = 100 

5 

IF 5/2G/60_abs_tstat_buy > 20, THEN set 5/20/60day_abs_tstat_buy = 20 
IF 5/20/60day_abs tstatjmy < .01, THEN set 5/2W60day_prob_buy = 0 
10 5day_prob_buy = lookup value indicated by using 5day_abs tstat_buy and 5day_df_buy 

20day_prob_buy = lookup value indicated by using 20day_abs_tstat_buy and 20day_df _buy 
60day_prob_buy = lookup value indicated by using 60day_abs_tstat_buy and 60day_df_buy 

15 

SteoS 

The conditional raw score may now be calculated. The conditional raw scare will be based on the 
average of the conditional raw scores for the 5 day period, 20 day period and for the 60 day period, 
which will then be multiplied by 100. The condition to determine whether the probability is added or 
20 subtracted is based on whether the average return is positive or negative, the formula is as follows 
(cond _ raw _ score _ buy )" 

[1 - (5 + or - Sday _ prob _ buy)] + [1 - (.5 + or - 2Qday _ prob , buy)] + fl - (5 + or - 6Qday _ prob _ buy)} 

3 

25 Logic: 

5 day portion of calculation 
IF analyst_company_total.avg_5dyjceturn_.buy > 0 
THEN subtract 5day_prob_buy from .5 in numerator 

30 

- This next case means that we have a negative average return for the events and we may add the 
probability 

ELSE add 5day_prob_buy to .5 in numerator 

35 
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20day portion of calculation 

IF analyst_company_totd_avg_20dy_retum buy > 0 

THEN subtract 20day_prob_buy from .5 in numerator 

5 - This next case means that we have a negative average return for the events and we may add the 
probability 

ELSE add 20day_prob_buy to .5 in numerator 

10 60day portion of calculation 

EF analyst„company - total.avg_60dy_retum_buy > 0 
THEN subtract 60day_prob_buy from .5 in numerator 

-This next case means that we have a negative average return for the events and we may add the 
15 probability 

ELSE add 60day_prob_buy to .5 in numerator 

Final step 

20 Take the average of the three calculated values above and multiply the result by 100 
Step 6 

Calculate the adjustment to the conditional raw score using the following formula (the adjustment is 
generally negative): 

25 

1 



adj _raw_buy = (-20) * 



ruim _5dy ^events _btq + num _20dy _events _buy + mm 

3 



- num_5dy_events_buy, num 20dy_events_buy and num_20dy_events_buy are taken from the 
analyst company__total table— 

30 

Note, the constant above may be changed for Analysts. 



Step 7 

Calculate the raw score by adding the adjustment to the conditional raw score: 
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raw_score_buy = (cond_raw - score_buy) + (adj_raw_buy) 

. here we arc adding a negative number (adj_jaw_sell), therefore the adjustment should generally be 
5 negative— 

Step 8 

new_raw_score_buy = raw_score_buy + (adj2_ raw_buy) 
10 Calculate second adjustment to raw_score__buy, variable name: adj2_raw_buy 
The adj2_jaw_buy is comprised of the following 4 components: 

(1) 5 day returns portion of calculation 

15 IF analyst_company_total.avg_5dy_retura_bny > 10 
THEN 5day_adj2_buy =10 

— This next case means that we have a 5 day return less or equal than 1000% and should use the 
following formula. Note the formula is raised to the third power which is equivalent to the cube root — 

20 

ELSE 5day_adj2_buy = (100 * analyst_company_total.avg_5dy_return_buy) 1/3 

(2) 20 day returns portion of calculation 

W analyst_company_total.avg_20dyjretum_buy > 10 
25 THEN 20day_adj2_buy = 10 

— This next case means that we have a 20 day return less than or equal 1000% and should use the 
following formula. Note the formula is raised to the third power which is equivalent to the cube root — 

30 ELSE 20day_ adj2_buy = (100 * analyst_company - total.avg_20dy„return_buy) l/3 

(3) 60 day returns portion of calculation 

IF analyst_company_total.avg__60dyreturn _buy > 10 
THEN 60day_adj2J>uy =10 

35 - This next case means that we have a 60 day return less than or equal 1000% and should use the 

following formula. Note the formula is raised to the third power which is equivalent to the cube root - 
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1/3 

ELSE 60day_ adj2_buy = (100* analyst_company_total.avg_60dy_return_buy) 
(4) Small Returns penalty 

IF analyst_companyjotal.avg_60dy_.return_buy >0.15 
5 THEN small_ret_buy_adj = 0 

— This next case means that we positive returns less than or equal 15%, so we will impose a penalty on 
the raw_buy_score - 

ELSE smaU_ret_buy_adj = (100* (0.15 - analyst_company_.total.avg_60dy_retum_buy)) 
Final step 

10 Combine the 4 components using the following formula to arrive at the final adj_raw: 

adj2 _raw_buy = (5day_buy_adj2 / 3) + (20day„buy_adj2 / 3) + (60day_buy_.adj2 / 3)- 
small_jet_buy_adj 

Step9 

Final step is to eliminate returns from the final scoring that have 60 day returns greater than 1000% 

15 IF analyst„company_totaLavg 60dy__return_buy > 10 
THEN filter this raw score in the final scoring process 

Step 10 

Analyst Scoring Logic - Ranked (Final) Score 

The following ranking system will be used to determine the final scores for each of the analyst scores: 

20 Important Note: Only analyst that cover companies within the Analyst Universe should be given a final 
Analyst score in this example. 

Analyst Bay Score 

Once all of the raw scenes have beed calculated, we can then rank the raw buy scores by sorting the 
results in descending order and separate this list into 100 groups. The group with the highest set of 
25 scores will be given the ranking of 100 (signifying the best performance), and the group with the lowest 
set of scores will be given the ranking of 1 (signifying the worst). The analysts in each group should 
then be assigned a score equal to the group that they are in. 

Step 1 Develop list of analysts who have received a raw_buy_score at companies within the 

Analyst Universe 

30 Step 2 Count the number of analysts in the list created in Step 1 



10 
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Step 3 Sort raw_score list descending for all analyst id/company_id where there is a score. 

[Note: If a secondary sort is needed, use analyst company_totaLavg.60day return descending]. 
Step 4 Use the look up table below to divide the ranked list into seven groups, and then within 
each group divide the ranked raw-score list into an equal number of observations for each score within 
the group (as defined in Table A shown below). 

Step 5 Record the score for the associated analystjd and company_id combinations in the group 

Raw Sell Analyst Score - Note, all inputs are based on calculated values related to earnings estimate 
revision changes 

For every analyst id and company-id combination in the analyst company total table with 2 or more 
earnings estimate revisions, calculate a raw score using the following logic (the logic is first 
summarized and then explained in detail): 

15 Step 1 Calculate the average 6 day, 20 day and 60 day returns for earnings revisions [5/20/60 
event returns] for each analyst/company combination (done by ABE) 

Step 2 Calculate the t -statistic [t-stat] values for 5/20/60 event returns in absolute terms 

Step 3 Calculate the degrees of freedom EDF] values for 5/20/60 event returns 

Step 4 Look up probability value using DF and [t-stat] for 5/20/60 event returns 

20 Step 5 Calculate raw score using conditional logic 

Step 6 Calculate adjustment to raw score 

Step 7 Calculate adjusted raw score 

Step 8 Calculate second adjustment to raw score and new raw score 

Step 9 Eliminate raw scores with returns greater than 1000% 

25 Step 10 Sort and rank the raw scores to produce the final Analyst Sell Scores. 

In this example, for every analyst id and company.id combination in the analyst_company_total table 
in the Analyst dB with analyst_company_totaI.events >= 2 perform the following logic: 

30 Stepl 

The 5 day, 20 day and 60 day return values are calculated by ABE for each analyst/company 
combination, 



35 



Average 5day return = analyst_company_total.avgJ5dy_retunuseU 
Average 20day return = analyst_companyj:otal.avg_20dy_retuni_sell 
Average 60day return = analyst_company_total.avg_60dy_retum u _sell 
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Step 2 

The t-slat needs to be calculated in absolute terms (Le., a non-negative number) for the 5 day, 20 day 
and 60 day returns using the following formulas: 

5 

The 5/20/6Oday_abs_tstat values may be rounded to 2 decimal places 



Before beginning this calculation, we must ensure that we will not be dividing by zero. If the standard 
deviation is zero, then we will set the final raw score to zero and move to the next analyst/company 
10 combination. 



IF analyst_company_Jotai.sd_5dy_sells = 0, THEN iaw_score = 0 
And start over with the next analyst/company buy combination 



15 IF analyst_company_total.sd_20dy_sells = 0, THEN raw_score sell = 0 
And start over with the next analyst/company buy combination 



20 



IF analyst - company_total.sd_60dy_sells = 0, THEN raw_score__sell = 0 
And start over with the next analyst/company buy combination 



Sday _abs _tstat _ sell = 



analyst _ company total Jtvg _ Sdy _ return sell 



analyst ^company _ total jsd _5dy _ sell ^ 
^ ^analyst _ company _ total jium _ Sdy _ event _ sell 



Iftday _ abs _ tstat _setl = 



analyst __ company total nvg _ 20dy _ return _ sell 



analyst ^company _ totaled _2Qrfy _ sell 
^ ^analyst _ company _ total Jtum _ 20rfy event ~ sell ^ 



25 60day _abs _tstat _seU = 



analyst ^company _ total javg _ 60dy _ return _ sell 



^ analyst __ company _ total jd 6Qdy _ sell 

^ tJ analyst _ company _ total aium _ 60dy _event _ sell j 
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Step 3 

The degrees of freedom are simply calculated by subtracting 1 from the number of observations for 
each return value: 

5 5dayjlf_seU =analyst_company-total. num_5dy_event_event_sell- 1 
20day_df_sell = analyst jco7npanyjtotaLnum_20dy_event_sell- 1 
60dayjdfjsell = ajialyst_company_totaLnunj60dy_everU_sell-l 
Step 4 

For each return period, lookup the probability using the absolute t stat and DF for each respective 
10 calculation as inputs to a lookup table: 

IF 5/2G/60day_df_sell > 100, THEN set 5/2(V60day_dLseIl = 100 

IF 5/20/60 abs_tstat_sell > 20, THEN set 5/2G/60day_abs_tstat_sell = 20 

15 

IF 5/20/60day_abs__tstat sell < .01. THEN set 5/2Q/60day_prob„sell = 0 
5day_prob_sell = lookup value indicated by using 5day_abs_tstat_sell and 5day_df_sell 
20 20day_prob_sell = lookup value indicated by using 20day_abs_tstat_sell and 20day_dfjsell 
60day_prob_sell = lookup value indicated by using 60day_abs_tstat_sell and 60day_df_sell 
StepS 

25 The conditional raw score may now be calculated The conditional raw score will be based on the 
average of the conditional raw scores for the 5 day period, 20 day period and for the 60 day period, 
which will then be multiplied by 100. The condition to determine whether the probability is added or 
subtracted is based on whether the average return is positive or negative, the formula is as follows 
( cond _ raw _ score _ sell )•* 

30 

P - (.5 + or - Sday _ prob _ sell)] + [l — {3 + or — 2Qday_prob_sell)] + [1 - (5 + or - today ^ prob_ sell)] 

3 

Logic: (Note: We are now using less than zero, which is opposite of the Buy logic) 
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5 day portion of calculation 

IF analyst_company total.avg_5dy_return_sell < 0 

THEN subtract 5day_prob from .5 in numerator 

5 

- This next case means that we have a positive average return for the events and we may add the 
probability 

ELSE add 5day_prob to .5 in numerator 

10 

20day portion of calculation 

IF analyst conq>any_total.avg_20dy_return_sell < 0 

THEN subtract 20day_prob from .5 in numerator 

15 - This next case means that we have a positive average return for the events and we need to add the 
probability 

ELSE add 20day_prob to .5 in numerator 

20 60day portion of calculation 

IF analyst_conq)any„total.avg_60dy__return sell < 0 
THEN subtract 60day_prob from .5 in numerator 

- This next case means that we have a positive average return for the events and we need to add the 
25 probability 

ELSE add 60day_prob to J in numerator 

Final step 

30 Take the average of the three calculated values above and multiply the result by 100 
Step 6 

Calculate the adjustment to the conditional raw score using the following formula (the adjustment will 
always be negative): 



35 
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num _Sdy _ events _ sell + num _ 20d> _ events _ sell + num _60dy ^events _seU 

3 



J 



- num_5dy_events sell, num_20dy_events_sell and num_60dy_events_sell are all taken from the 
analyst_company_total table- 

Step 7 

Calculate the raw sell scene by adding the adjustment to the conditional raw score: 
raw_score_seIl = (cond_raw_score_sell) + (adj_raw_sell) 



- here we are adding a negative number (adj_raw_sell), therefore the adjustment should generally be 
negative — 

Step 8 

15 new_raw_score_sell = raw_score_sell + (adj2_jraw„sell) 

Calculate second adjustment to raw_score_se!l, variable name: adj2__raw_sell 
The adj2 raw is comprised of the following 4 components: 



(1)5 day returns portion of calculation 

IF analyst company_totaLavg_5dy_return_sell > 10 

THEN 5day_sell_adj2 = 10 

25 - This next case means that we have a 5 day return less or equal than 1000% and should use the 

following formula. Note the formula is raised to the third power which is equivalent to the cube root - 

* 

ELSE 5day_sell_adj2 = (100 * andyst_company_total.avg_5dy_return„sell) 1/3 

30 (2) 20 day returns portion of calculation 

IF analyst_company_total^vg_20dy„return_sell > 10 
THEN 20day„sell_adj2 = 10 



10 



20 
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- This next case means that we have a 20 day return less than or equal 1000% and should use the 
following formula. Note the formula is raised to the third power which is equivalent to the cube root - 

ELSE 20day_selLadj2 = (100 * analyst_company_total.avg_20dy_retum_seU)^ 

5 

(3) 60 day returns portion of calculation 

IF analyst_company_total.avg_60dy_return_sell > 10 

THEN 60day^selLadj2 = 10 

10 - This next case means that we have a 60 day return less than or equal 1000% and should use the 

following formula. Note the formula is raised to the third power which is equivalent to the cube root - 

ELSE 60day_sell_adj2 = (100 * analyst_company_totaLavg_60dy_return_sell) 1/3 

1 5 (a) Small Returns penalty 

IF analyst_cximpany_total.avg_60dy_return_sell < -0.08 
THEN small _jet_sell_adj = 0 

- This next case means that we negate returns less than or equal -8%, so we will impose a penalty on the 
20 raw _sell__score - 

ELSE small_ret_selLadj = (100 * (-0.08 - analyst_company„total.avg_60dy_return_sell)) 
Final step 

25 Combine the 4 components using the following formula to arrive at the final adj2_raw_sell: 

adj2_raw_sell = (5day_sell_adj2/3) + (20day_sell_adj2/3) + (60day_adj2/3) + small_ret_sell_adj 
Step 9 

30 Final step is to eliminate returns from the final scoring that have 60 day returns greater than 1000% 

IF analyst_con5>any_total.avg_60dy_return_sell > 10 
THEN filter this raw score in the final scoring process 
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Step 10 

Analyst Scoring Logic - Ranked (Final) Score 
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The following ranking system may be used to determine the final scores for each of the analyst scores: 

Important Note: Only analysts thai cover companies within the Analyst Universe should be given a 
final Analyst score in this example 

5 

Analyst Sell Score 

Once all of the raw sell scores have been calculated, we can then rank the raw sell scores by sorting the 
results in descending order and separate this list into 100 groups. The group with the highest set of 
scores will be given the ranking of 100 (signifying the best sell performance), and the group with the 
10 lowest set of scores will be given the ranking of 1 (signifying the worst sell performance). The analysts 
in each group should then be assigned a score equal to the group that they are in. 

Step 1 Develop list of analysts who have received a raw_sell_score at companies within the Analyst 
Universe 

IS Step 2 Count the number of analysts in the list created in Step 1 

Step 3 Sort raw_sell_scare list descending for all analyst_id/company_id where there is a score. 

[Note: If a secondary sort is needed, use analyst company_total.avg_60dy_retum_sell descending]. 

Step 4 Use the look up table shown below to divide the ranked first into seven groups, and then 

within each group divide the ranked raw_sell score list into an equal number of observations for each 
20 score within the group (as defined in the table shown below). 

Step 5 Record the score for the associated analyst Jd and companyjd combinations in the group 

LookUp Table A 

25 

This look up table provides the information needed to assign the Analyst Buy Score and Analyst Sell 
Score for each analyst 

Using the rankings determined in Step 3 of Analyst Buy Score/Analyst Sell Score procedures derive the 
30 Analyst Buy Score and Analyst Sell Score split the rankings into the 7 groups in the table below. 



Group 


Percentages of Total 


Analyst Buy Score 


Analyst Buy Score 




Observations 


Calculation 





wo 
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1 


Top 5.0% (rounded 
up to the nearest 
whole number) 


Divide the top 5% of 
total observations into 
10 groups with an 
equal number of 
observations in each 
group 


91 to 100 


2 


Next 12.5% (Le. read 
sorted records until 
reach 17.5% of the 
total number o 
individuals from Step 
1 of the scoring 
procedures) 


Divide the 12.5% of 
total observations into 
15 groups with an 
equal number of 
observations in each 
group 


76 to 90 


3 


Next 18.0% (Le. read 
sorted records until 
reach 35.5% of the 
total number of 
individuals from Step 
1 of the scoring 
procedures) 


Divide the 18.0% of 
total observations into 
15 groups with an 
equal number of 
observations in each 
group 


61 to 75 


4 


Next 29.0% (i.e. read 
sorted records until 
reach 64.5% of the 
total number of 
individuals from Step 
1 of the scoring 
procedures) 


Divide the 29.0% of 
total observations into 
20 groups with an 
equal number of 
observations in each 
group 


41 to 60 


5 


Next 18.0% (Le. read 
sorted records until 
reach 82.5% of the 
total number of 
individuals from Step 
1 of the scoring 
procedures) 


Divide the 18.0% of 
total observations into 
15 groups with an 
equal number of 
observations in each 
group 


26 to 40 
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6 


Next 12.5% (Le. read 


EH vide the 12.5% of 


11 to 25 




sorted records until 


total observations into 






reach 95.0% of the 


15 groups with an 






total number of 


equal number of 






individuals from Step 


ouservauons in eacn 






1 of the scoring 


group 






procedures) 






7 


Last 5.0% (rounded 


Divide the last 5% of 


1 to 10 




up to the nearest 


total observations into 






whole number) (i.e. 


10 groups with an 






read sorted records 


equal number of 






until reach, 100.0% of 


observations in each 






the total number of 


group 






in/I'! \sirltialc finr^f i ^t^n 
lUUlVlUUcLld 1J.UIXI i3icp 








1 of the scoring * 








procedures) 







In accordance with the principles of the present invention, an analyst's performance (or other 
investment professional or investment advisor) is evaluated by utilizing information pertaining to at 
least one revision made by the analyst involving at least one investment This evaluation includes 
5 determining a conditional performance score indicative of the analyst's performance relative to other 
analysts. The performance score is determined, at least in part, by considering an average historical 
performance of the investment following the revision. In addition* the performance score is also 
determined using a historical consistency of the analyst's performances with respect to revisions or 
reports involving the investment, and a number of revisions, recommendations, or opinions made by the 
10 analyst Then* the performance score may be adjusted according to one or more adjustments to 
generate a final performance score. 

These performance scores iflay be determined based on analyst recommendations, such as 
upward or downward revisions or a combination of both. In particular, a score based on upward 

15 revisions, or an upward revision performance score, corresponds to the predictive nature of an analyst 
with respect to predicting or influencing increases in earning. In contrast, a score based on downward 
revisions, or a downward revision performance score, corresponds to the predictive nature of an analyst 
with respect to predicting or influencing decreases in earnings. Similarly, a combined score or, for 
example, an average of the two previous scores, corresponds to the overall predictive nature of the 

20 analyst 
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Hence, the present invention provides a method, system, and computer readable medium 
storing computer executable instructions for evaluating an analyst's performance by utilizing 
information pertaining to at least one revision made by the analyst involving at least one investment In 
5 one embodiment, the present invention includes determining a conditional performance score indicative 
of the analyst's performance relative to other analysts. This conditional performance score is 
determined at least in part by considering an average historical performance of the at least one 
investment following the at least one revision, a historical consistency of the analyst's performances 
with respect to revisions involving the at least one investment, and a number of revisions made by the 
10 analyst In addition, the present invention includes adjusting the conditional performance score of the 
analyst according to a return amount adjustment to generate a final performance score. 

hi another embodiment, the present invention provides a method, system, and computer 
readable medium storing computer executable instructions for evaluating an analyst's performance. 

15 This embodiment includes utilizing information pertaining to at least one revision made by the analyst 
involving at least one investment, including an upward or downward change in opinion of the analyst 
with regard to at least one investment In addition, the present invention includes calculating a raw * 
conditional performance score indicative of the analyst's performance relative to other analysts, said 
raw conditional performance score determined at least in part by considering a measure of variability of 

20 the analyst's performance, an average historical performance of the at least one investment following 
the at least one revision, a standard deviation of the at least one revision* a number of revisions made by 
the analyst, and a likelihood that the at least one revision will actually produce an expected result 
From there, the invention includes adjusting said raw conditional performance score according to a 
return amount adjustment to produce a final performance score. 

25 

In another embodiment, the present invention provides a method of evaluating an analyst's 
performance by utilizing information pertaining to at least one revision made by the analyst involving at 
least one investment hi this embodiment, the invention includes calculating a raw conditional 
performance score indicative of the analyst's performance relative to other analysts. The raw 

30 conditional performance score may be determined at least in part by considering a measure of 
variability of the analyst's performance, an average historical performance of the at least one 
investment following the at least one revision, a standard deviation of the at least one revision, a 
number of revisions made by the analyst, and a likelihood that the at least one revision will actually 
produce an expected result Subsequently, the invention adjusts the raw conditional performance score 

35 according to a modifier accentuating the number of revisions made by the analyst In addition, the raw 
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conditional performance score may be adjusted according to a small return penalty penalizing the 
analyst for small returns to produce a final performance score. 

In still yet another embodiment, the present invention includes a method, system, and computer 
5 readable medium storing computer executable instructions for evaluating an analyst's performance, and 
includes utilizing information pertaining to at least one upward or downward revision including an 
upward or downward change in opinion of the analyst with regard to at least one investment This 
embodiment also includes calculating a performance score indicative of the analyst's performance 
relative to other analysts, said performance score determined at least in part by considering a measure 

10 of variability of the analyst's performance, an average historical performance of the at least one 

investment following the at least one upward or downward revision, a standard deviation of the at least 
one upward or downward revision, a number of upward or downward revisions made by the analyst, 
and a likelihood that the at least one upward cm* downward revision will actually produce an expected 
result In addition, this embodiment includes comparing the analyst's performance score against 

IS performance scares of other analysts to produce at least one of an upward, downward or combined 
revision ranking. 

In yet another embodiment, a method of evaluating an analyst's performance by utilizing 
information pertaining to at least one revision made by the analyst involving at least one investment is 

20 disclosed. In this embodiment, the invention includes determining a first average return value (ave. 
returnO, second average return value (ave. return^, and third average return value (ave. return 3 ) for the 
at least one revision. Each of the first, second, and third average return values corresponds respectively 
to an average rate of return on the at least one revision involving the investment for a first period of 
time (tinrei), a second period of time (time^, and a third period of time (times). From there, a first t-stat 

25 value (t-statO for ave. return lt a second t-stat value (t-stat;t) for ave. return^ and a third t-stat value (t- 
stat 3 ) for ave. return 3 , are calculated, in absolute terms. In particular, the first, second, and third t-stat 
values are calculated by utilizing ave. returni, ave. return^ ave. return 3 , a first standard deviation (SDO, 
a second standard deviation (SD2), and a third standard deviation (SD3), a total number of revisions 
(revisions^ occurring more than or tfqual to timei before said evaluating, a total number of revisions 

30 (revisions^ occurring more than or equal to times before said evaluating, and a total number of 

revisions (revisions 3 ) occurring more than or equal to times before said evaluating. Subsequently, a 
first degree of freedom (DFi) for ave. returni, a second degree of freedom (DF2) for ave. retum 2 , and a 
third degree of freedom (DF 3 ) for ave. return 3 are calculated. The invention then contemplates 
determining a first probability (probi) for timet by utilizing t-stati and DFi, a second probability (prob^ 

35 for time2 by utilizing t-stat 2 and DF 2 , and a third probability (prot^) for time 3 by utilizing t-stat 3 and 
DF 3 . A raw conditional performance score for the analyst is generated by utilizing probi, prot>2, prol^. 
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A conditional performance score (cond performance scene) may be generated by adding a modifier 
(adjustment), which may be determined by utilizing revisions!, revisions and revisions 3 , to the raw 
conditional performance score. From there, the invention contemplates generating a small return 
penalty by subtracting a penalty from an average of return factors corresponding to time lv time2, and 
5 ture 3i wherein for upward revisions the penalty is 0 if ave. retum 3 is greater than 15%, else the penalty 
is equal to 15% minus ave. return 3 , wherein for downward revisions the penalty is 0 if ave. return 3 is 
less than -8%, else said penalty is equal to -8% minus ave. return 3f wherein the first period of time 
return factor is ten if ave. return] is greater than ten, else the first period of time return factor is equal to 
a cube root of a product of 100 and ave. return!, wherein the second period of time return factor is ten if 

10 ave. return 2 is greater than ten, else the second period of time return factor is equal to a cube root of a 
product of 100 and ave. return?, and wherein the third period of time return factor is ten if ave. return 3 is 
greater than ten, else the third period of time return factor is equal to a cube root of a product of 100 and 
ave. return 3 . Using the above factors, a final performance score (performance score) indicative of the 
analyst's performance may be generated by adding the small return penalty to the conditional 

15 performance score. 

There has thus been outlined, rather broadly, the more important features of the invention in 
order that the detailed description thereof that follows may be better understood, and in order that the 
present contribution to the art may be better appreciated. There are, of course, additional features of the 
20 invention that will be described hereinafter and which will form the subject matter of the claims 
appended hereto. 

In this respect, before explaining at least one embodiment of the invention in detail, it is to be 
understood that the invention is not limited in its application to the details of construction and to the 
25 arrangements of the components set forth in the following description or illustrated in the drawings. 
The invention is capable of other embodiments and of being practiced and carried out in various ways. 
Also, it is to be understood that the phraseology and terminology employed herein are for the purpose 
of description and should not be regarded as limiting. 

30 As such, those skilled in the art will appreciate that the conception, upon which this disclosure 

is based, may readily be utilized as a basis for the designing of other structures, methods and systems 
for carrying out the several purposes of the present invention. It is important, therefore, that the claims 
be regarded as including such equivalent constructions insofar as they do not depart from the spirit and 
scope of the present invention. 
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Further, the purpose of the foregoing abstract is to enable the U.S. Patent and Trademark Office 
and the public generally, and especially the scientists, engineers and practitioners in the art who are not 
familiar with patent or legal terms or phraseology, to determine quickly from a cursory inspection the 
nature and essence of the technical disclosure of the application. The abstract is neither intended to 
5 define the invention of the application, which is measured by the claims, nor is it intended to be limiting 
as to the scope of the invention in any way. 

These together with other objects of the invention, along with the various features of novelty 
which characterize the invention, are pointed out with particularity in the claims annexed to and 
10 forming a part of this disclosure. For a better understanding of the invention, its operating advantages 
and the specific objects attained by its uses, reference should be had to the accompanying drawings and 
descriptive matter in which there is illustrated preferred embodiments of the invention. * 

Other objects of the present invention will be evident to those of ordinary skill, particularly 
15 upon consideration of the following detailed description of the preferred embodiments. 

NOTATIONS AND NOMENCLATURE 

The detailed descriptions which follow may be presented in terms of program procedures 
20 executed on computing or processing systems such as, for example, a stand-alone gaming machine, a 
computer or network of computers. These procedural descriptions and representations are the means 
used by those skilled in the art to most effectively convey the substance of their work to others skilled 
in the art 

25 A procedure is here, and generally, conceived to be a self-consistent sequence of steps leading 

to a desired result These steps are those requiring physical manipulations of physical quantities. 
Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable 
of being stored, transferred, combined, compared and otherwise manipulated. It proves convenient at 
times, principally for reasons of common usage, to refer to these signals as bits, values, elements, 

30 symbols, characters, terms, numbers, or the like. It should be noted, however, that all of these and 
similar terms are to be associated with the appropriate physical quantities and are merely convenient 
labels applied to these quantities. 



35 



Further, the manipulations performed are often referred to in terms, such as adding or 
comparing, which are commonly associated with mental operations performed by a human operator. 
No such capability of a human operator is necessary, or desirable in most cases, in any of the operations 
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described herein which form part of the present invention; the operations are machine operations. 
Useful machines for performing the operation of die present invention include general purpose digital 
computers or similar devices. 

RttTCT ntt SCRIPTION OF THE DRAWINGS 

FIG. 1 illustrates a prior art stock and cash portfolio development system; 

FIG. 2 illustrates a prior art securities valuation system; 

FIG. 3 illustrates a prior art investment fund management system; 

FIG. 4 illustrates a prior art data processing system for selecting securities and constructing 
investment portfolios based on a set of artificial neural networks; 

FIGS. 5a-5b depict one example of a user interface utilizable for displaying an analyst 
evaluation according to the principles of the present invention; 

FIGS. 5c-5d depict one example of a user interface displaying examples of revisions utilizable 
in conjunction with the principles of the present invention; 

FIG. 6 is a flow diagram illustrating a process for evaluating analysts according to the 
principles of the present invention; 

FIG. 7 is a flow diagram illustrating a high level process for generating an upward revision 
performance measure utilizable in the process of FIG. 6; 

FIG. 8 is a flow diagram illustrating a high level process for generating a downward revision 
performance measure utilizable in the process of FIG. 6; 

FIG. 9 is a flow diagram illustrating a process for generating an upward revision small return 
adjustment utilizable in the processes of FIG. 7; 

FIG. 10 is a flow diagram illustrating a process for generating a downward revision small 
return adjustment utilizable in the processes of FIG. 8; 

FIG. 11 is a flow diagram illustrating a process for ranking analysts according to an upward 
revision performance measure utilizable in the process of FIG. 7; 

FIG. 12 is a flow diagram illustrating a process for ranking analysts according to a downward 
revision performance measure utilizable in the process of FIG. 8; 

FIG. 13 illustrates one example of a central processing unit for implementing a computer 
process in accordance with a computer implemented embodiment of the present invention; 

FIG. 14 illustrates one example of a block diagram of internal hardware of the central 
processing unit of FIG. 13; 

FIG. IS illustrates another example of a block diagram of internal hardware of the central 
processing unit of FIG. 13; 
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FIG. 16 illustrates one example of a memory medium which may be used for storing a 
computer implemented process of the present invention; 

FIG. 17 illustrates an example of a combined internet, POTS, and ADSL architecture which 
may be used with the present invention; and 
5 FIG. 18 shows a set of tabulated results of an evaluation performed according to the principles 

of the present invention. 

BEST MODE FOR CARRYING OUT THE INVENTION 

10 Reference now will be made in detail to the presendy preferred embodiments of the invention. 

Such embodiirents are provided by way of explanation of the invention, which is not intended to be limited 
thereto. In fact, those of ordinary skill in the art may appreciate upon reading the present specification and 
viewing the present drawings that various modifications and variations can be made. 

15 Far example, features illustrated or described as part of one embodiment can be used on other 

embodiments to yield a still further embodiment Additionally, certain features may be interchanged with 
similar devices or features not mentioned yet which perform the same or similar functions. It is therefore 
intended that such modifications and variations are included within the totality of the present invention. 

20 In accordance with the principles of the present invention, an analyst' s performance is 

evaluated by utilizing information pertaining to at least one revision made by the analyst involving at 
least one investment This evaluation includes determining a conditional performance score indicative 
of the analyst's performance relative to other analysts. The performance score is determined at least in 
part by considering an average historical performance of the investment following the revision. In 

25 addition, the performance score is also determined using a historical consistency of the analyst's 

performances with respect to revisions involving the investment, and a number of revisions made by the 
analyst Then, the performance score may be adjusted according to one or more adjustments to 
generate a final performance score. 

* 

30 These performance scores may be determined based on upward or downward revisions or a 

combination of both. In particular, a score based on upward revisions, or an upward revision 
performance score, corresponds to the predictive nature of an analyst with respect to predicting or 
influencing increases in earning. In contrast, a score based on downward revisions, or a downward 
revision performance score, corresponds to the predictive nature of an analyst with respect to predicting 

35 or influencing decreases in earnings. Similarly, a combined score, or for example an average of the two 
previous scores, corresponds to the overall predictive nature of the analyst 
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Referring first to FIGs. 5a and 5b, one example of a user interface 500 utilizable for displaying, 
for example, analyst performance scores is illustrated. In FIG. 5, the user interface displays a wide 
variety of information including performance scores for any number of companies 510. To calculate 
5 the company performance scores in this example, the scores for each of the analysts covering a 
particular company are averaged to produce an overall company score. In addition to listing and 
determining an average of the performance scores of all the analysts covering a company, any 
combination of performance scores may optionally be determined and listed. For instance, an average 
of only the upward or downward revision performance scores for the analysts covering a particular 
10 company may be determined and listed. Alternatively, a user may just as easily elect to view a list of 
only the upward or downward revisions for a single company. Similarly, an index of individual analyst 
performance scores, independent of the companies, based either on upward or downward revisions or 
both may also be determined and listed. 

15 Advantageously, these performance scores are a measure of the predictive nature of each 

analyst's revisions. In one example, a revision is a change in the opinion of an analyst on a 
performance of a company or other entity, for example, overall company revenues, profits, market 
share, and/or how much a company is likely to earn per share in a future quarter, fiscal year, or other 
time period. For instance, as shown in FIGs. 5c-5d, other examples of revisions utilizable with 

20 embodiments of the present invention include detailed analyst recommendations on whether to sell, 
hold or buy a security 530, earnings surprises 532, estimates submitted 534, price/earnings 536, 
momentum, price/earning ratio, P/E and growth rates, and/or PEG ratio, and/or other similar 
information/estimates. Some causes for an upward revision include: the release of a good quarterly or 
annual earnings report, a positive analysts' meeting or conference call with company executives, or 

25 improvements at a similar company in the same industry. Likewise, causes for a downward revision 
include; the release of a grim quarterly or annual earnings report, a negative analysts' meeting or 
conference call with company executives, or worsening results at a similar company in the same 
industry. 

* 

30 In the present invention, the higher the score of the analyst (with one-hundred, for example, 

being the highest), the more predictive on a historical basis. Furthermore, the performance score 
increases with the number of beneficial revisions of the analyst For instance, the performance score 
increases when a rate of return of an investment increases and the revision is an upward revision, or 
when a rate of return of the investment decreases and the revision is a downward revision. Likewise, 

35 the performance score decreases when a rate of return of the investment decreases and the revision is an 
upward revision, or when a rate of return of the investment increases and the revision is a downward 
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revision. Thus, a high upward revision performance score corresponds to historically high positive 
returns (e.g., the analyst issued an upward revision, and the stock went up in value). Similarly, a high 
downward revision performance score corresponds to high negative returns (e.g., the analyst issued a 
downward revision, and the stock wait down in value). In addition, it should be noted that although in 
S this particular embodiment corporate securities are mentioned as being one example of the analyst's 
revisions, the instant invention is also applicable to and should be construed as encompassing any other 
form of similar investment Some examples include bonds, currencies, commodities, paper, precious 
metals, minerals, natural resources, mutual funds, and the like or any other analogous investments 
and/or investment entities. 

10 

As will be discussed in greater detail below, several factors are incorporated into these 
performance scores. For instance, an analyst's average return history (e.g., a return amount typically in 
a percentage calculated over a predetermined period of time after the analyst's revision) for both 
upward and downward forecast revisions may initially be considered. Indeed in alternative 

IS embodiments, this factor is accorded the greatest amount of weight Next, the analyst* s historical 
consistency at issuing upward and downward revisions in a company's stock, as measured by, for 
example, standard deviation and the like are also considered. In a related maimer, a measure of 
variability of the analyst's performance may also be considered. Similarly, a likelihood that the 
revision will actually produce an expected result, as embodied by statistical characteristics such as 

20 probability and the like are also factored into the calculation. Finally, the number of revisions that 
make up an analyst's track record is factored into the performance score for the simple reason that the 
more times an analyst has demonstrated good judgment the better. Other factors are optionally used as 
well. 

25 As to the analyst's average return history, any number of return periods may be considered. 

For example, in one embodiment, two periods are utilized. In other embodiments, three or more 
periods may be used. Furthermore, any length of time is possible. Thus, in some cases, two periods 
(e.g., a relatively short period of time and a relatively long period of time) suffices. In other cases, one 
or more intermediate periods of timtf may also be included into the calculation. As examples, three 

30 month and six month periods may be used. As another example, five, twenty, and sixty day periods 
may be used. Similarly, one week, one month, and two month periods are also possible. 

In addition to the above, other factors may optionally be considered as well. For example, the 
above factors may be utilized to generate a conditional or raw performance score, which in turn may be 
35 modified or adjusted to produce a final performance score. For instance, a modifier accentuating the 
number of revisions made by the analyst may be factored into the conditional performance score. In 
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other cases, a return amount adjustment may be made to reward or penalize an analyst for small or large 
return amounts. For example, the return amount adjustment may include a small return penalty 
penalizing or reducing the analyst's performance scene for returns less than a predetermined amount 
More specifically, a penalty may be imposed for returns of less than a predetermined amount as 
5 calculated over a predetermined period of time. Hence, with upward revisions, a penalty might be 
imposed for returns of less than approximately 15% as calculated over approximately a two month 
period of time. In contrast, with downward revisions, a penalty might be imposed for returns greater 
than --8% as calculated over a two month period of time. Likewise, high returns may also result in 
increases to the analyst's performance score. Further, adjustments need not be limited to return 
10 amounts. As an example, adjustments may be made to a performance score for an extremely large 

number of accurate trades. Indeed, the present invention contemplates modifications or adjustments for 
nearly any factor utilizable for predicting accuracy or reliability. 



Returning to the interface displayed in FIG. S, in addition to listing the average scores for all of 
IS the analysts associated with a particular company, the interface may also be used to list each of the 
highest performing analysts for categories pertaining to downward and upward performance scores. 
Furthermore, interface 500 may also be linked to or used to display information concerning each 
analyst and/or the corresponding security. Likewise, the interface sponsor may optionally include 
commentary concerning, for example, any securities or analysts, advertisements utilizable as a source 
20 of revenue, instructions or help information for beginning users, and information concerning or tracking 
the activities of exceptionally high performing analysts, or other similar features. 

While the interface depicted in FIG. 5 is used to view the average of the performance scores 
associated with a particular list of companies, other listings are also possible. For example, a user may 

25 select a particular analyst for review. In this case, each of the upward, downward, and combined 

performance scores for that analyst may be viewed. Furthermore, if the selected analyst is responsible 
for monitoring more than one company, multiple values for upward, downward, and combined 
performance scores (e.g., one score for each company) may be listed. Similarly, a user may select any 
combination of analysts or sets/groupings of data including, for example, all analysts dealing with or 

30 investing in a particular industry or market, or other analogous listings of information. 

A broad overview of the process of one embodiment of the present invention for calculating 
performance scores is now illustrated with reference to FIG. 6. First, a list of analysts is retrieved from 
one or any number of databases 610. As will be discussed in greater detail below with reference to 
35 FIG. 17, the database may include a list of any or all of a variety of combinations of analysts. For 
example, all of the analysts tracking companies in a single industry may be included. Similarly, the 
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database may include all insiders filing insider documents with the SEC. In other embodiments, other 
similar or analogous techniques may be utilized to generate a list of analysts. For example, any one of a 
variety of databases tracking industry leaders may just as easily be used 

After retrieving the list of analysts, a smaller list of analysts to be evaluated is optionally 
generated according to predetermined criteria. In this regard, a check is performed identifying analyst 
that meet predetermined criteria 6 IS. In this example, any number of criteria may be used to generate 
this list For example, each analyst may be required to be an actual living person, thereby excluding 
brokerage firms. Thus, in this scenario, any entities that are not people, such as companies, 
partnerships, or trusts, are removed from the evaluation list As another example, the market 
capitalization of the company being monitored may be required to be greater than a predetermined 
amount, such as, for instance, $50 million. Third, each analyst may be required to have issued a 
minimum number of revisions that have been in existence for a certain period of time. For instance, 
some embodiments may require analysts to have issued at least two revisions, with each revision being 
at least sixty days old. Other embodiments may require more or fewer revisions, which in turn may be 
more recent or older than sixty days. These requirements are disclosed for exemplary purposes only 
and it is to be understood that other types of analyst requirements are possible within the scope of the 
invention. For instance, non-people analysts such as partnerships, corporations, and other joint ventures 
or entities are also possible where groups of analysts participate in recommendations and/or decision 
making. Thus, a group of, for example, Morgan Stanley analysts may be compared against Goldman- 
Sachs analysts, individually or overall. Similarly, entities not having market capitalization or trading 
limits are also possible. Furthermore, the concepts of the present invention are applicable to entities 
investing in investments other than securities, such as bonds, mutual funds, or any other similar 
investments, trading securities, investments and the like. 

If the analyst does not meet the predetermined criteria (615), a message is returned, and the 
analyst is not evaluated 620. On the other hand, if the analyst meets the predetermined criteria, 
processing continues with the calculation of a performance score for the analyst 625. As will be 
discussed below, this performance scene may reflect upward revisions, downward revisions, or a 
combination, and/or average distribution of the two. From there, processing continues with a 
determination of whether more analysts exist to be ranked 630. 

If more analysts are to be ranked, processing returns to 610. If the list of analysts to be ranked 
is complete, each of the analysts meeting the predetermined criteria are ranked 635. Subsequently, 
these rankings may optionally be disseminated 640 via, for example, a user interface similar to that 
depicted in FIG. 5 in a local or global setting, e.g., intranets, the Internet, and the like. Alternatively, 
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Che rankings may be disseminated in any other similar or analogous medium, such as in a magazine or 
newspaper a trade journal, television, and radio or the like. 

Although in this embodiment a list of analysts to be evaluated is determined according to 
S predetennined criteria, it is to be understood that other alternatives are also possible. For instance, the 
present invention is to be construed as including scenarios where a user enters or selects a 
predetermined listing of analysts for evaluation. In such a case, only the particular entities entered by 
the user would be evaluated. It is also possible that a mixed grouping of investment entities and types 
may be evaluated. For instance, it is possible within the scope of the present invention to evaluate 
10 corporate insiders, nonperson joint ventures, brokerage firms, and/or corporations, and any other types 
of investment entities or combinations thereof. 

The calculation of an upward revision performance score is now described with reference to 
FIG. 7. In this example, one or more rates of return on an investment corresponding to an analyst's 

IS upward revision, taken a predetermined period of time after issuance of the upward revision, are 

utilized in the generation of the performance score. Thus, one or more return values for one or more 
return intervals are initially calculated or gathered for each analyst 705 by any suitable means. In this 
particular embodiment, three return values, one for a short period of time, one for an intermediate 
period of time, and one for a long period of time, are taken. As to the actual amount of time associated 

20 with each period, any lengths may be used. For example, the short period of time may be five days or a 
week, the intermediate period may be twenty days or a month, and the long period may be sixty days or 
two months. Furthermore, three return values are utilized in this embodiment for exemplary purposes 
only. In actual practice any number of return values, at any number of time intervals, are allowed so 
long as they are suitable proxies for, e.g., short, long, and optionally intermediate periods of time. As 

25 an example, two return values, one at 3 months (e.g., 13 weeks) and a second at 6 months (e.g., 26 
weeks), may just as easily be utilized. 

In accordance with the principles of the present invention and as discussed briefly above, these 
scores or measures reflect an analyst's performance and are generated by considering a number of 

30 performance factors. One embodiment of the present invention uses a measure of the analyst's 

historical performance as determined by rates of return over certain periods of time after a revision. 
Another example may include considering the analyst's success at selecting entry and exit points in the 
investment Additionally, as will be discussed below, the analyst's performance score may be modified 
or weighted based on a number of revisions made by the analyst, the magnitude of a return, or other 

35 similar factors. Of course, the above factors are listed for exemplary purposes only and it is to be 
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understood that other similar and analogous factors may also be considered so long as they provide an 
accurate and/or estimated measurement of the analyst's success or lack thereof. 



10 



After calculating the return values, a statistical measure of confidence in the analyst's 
performance may be calculated for each return value 710. As one example, a t-statistic or t-stat, taken 
in absolute terms, is utilized as this measure of confidence. Hie t-stat is a measure on a random sample 
(or pair of samples) in which a mean (or pair of means) appears in the numerator, and an estimate of the 
numerator's standard deviation appears in the denominator. The latter estimate is based on the 
calculated sample variance estimates of the samples. Alternatively, other standard variance measures or 
other suitable indices of variability may optionally be used in place of standard deviation, and/or t-stat 
If these calculations yield a value of (t) that is sufficiently different from zero, the test is considered to 
be statistically significant 



15 



hi this embodiment, the t-stats for each return period may be determined by: 

upward revision return 



upward revision t stat - 



upward _ revision _ SD 



^ total _ number _ of _ upward _ revisions ^ 
In this example since three intervals are being utilized, three t-stat values are calculated, one for each 



interval. 



To prevent division by zero, if the standard deviations for any of the time intervals are zero, 
then the conditional upward revision performance score for that interval is set to zero. In addition, 

20 although the t-stat is used in this example, other measures of confidence may just as easily be 

incorporated in the present invention. As another example, the z-statistic or other similar measurements 
of variability, dispersion, or consistency may be used Furthermore, regardless of what method is used 
to produce the measure of confidence, the measure of confidence may also be manipulated or weighted 
to consider other factors, so long as it accurately reflects or estimates, for instance, the analyst's return 

25 history and historical consistency at issuing forecast revisions in the investment 

After calculating the measures of confidence (e.g., the t-stats for each time interval or return 
period), one or more degrees of freedom are determined by subtracting, for example, one from the 
number of revisions for each return value 715. Hence, with three return values, three degrees of 
30 freedom are generated. The degrees of freedom are used to describe the number of values in a final 
calculation of a statistic that are free to vary, and measure, for example, repetition, repeatability, and 
certainty. Hence, each degree of freedom is determined by: 



WO 01/80124 



PCT/USO 1/11691 



33 

upward revision degree of freedom = total number of upward revisions - 1 

Next, a probability indicating, for example, the likelihood that an analyst's and/or other 
investing entity's actions will actually produce the expected result, is determined for each return 
interval 720. Thus, with three return intervals, three probabilities are generated In this example, the 
degrees of freedom, along with the absolute t-stats, are used in determining a probability for each return 
period 720. Specifically, subject to the following optional conditions, the probabilities are looked-up in 
a common statistics lookup table using each return interval's degree of freedom and t-stat If any of the 
degrees of freedom are greater than 100, then that degree of freedom is set to 100 before looking up the 
probability. If any of the absolute t-stats are greater than 20, then that absolute t-stat is set to 20 before 
looking up the probability. Finally, if any of the absolute t-stats are less than 0.01, then the 
corresponding probability is simply set to 0. hi addition, the probabilities may be calculated utilizing 
other methods. For instance, the t-stat and the degree of freedom may undergo further manipulation 
before looking up the probability. Alternatively, other analogous methods may be utilized in place of 
the look-up table to determine the probabilities. 



Subsequent to determining the probabilities for each return period, a conditional upward 
revision performance score, or raw conditional upward revision performance score, is generated 725. 
Although other methods are possible, the conditional performance score in this example is determined 
by: 



cond _up _ per _ score = 



[l-(5 + or-up_ probl)] + [l - (.5 + or - up _ prob2)\ + [l - (.5 + or - up __ prob3)} 



3 



In this formula, up_probl, up_prob2, up_prob3 correspond respectively to the probability for a first, 
second, and thud return interval. Referring again to die above formula, whether the probabilities are 
added or subtracted is determined by the following conditions. If any of the three probabilities are 
greater than zero, then that probability is subtracted from 0.5. Otherwise, 0.5 is added to the 
probability. 

The conditional upward revision performance score represents an initial analyst performance 
measure with respect to upward revisions and may undergo one or more of any number of 
manipulations to result in various other refined performance measures and ultimately a final upward 
revision performance score. As an example, the conditional performance score may be weighted to 
place additional emphasis on the maturity of the corporation. Alternatively, investments in, for 
example, conservative sectors may also receive a modification. To more clearly illustrate, exemplary 
modifiers accounting for the size of a return and a number of revisions issued are discussed in greater 
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detail below. These modifications are introduced for exemplary reasons only and it is to be understood 
that other manipulations are also possible within the scope of the present invention. 

Initially, an optional upward revision adjustment may be generated to reflect and emphasize the 
5 number of revisions issued by the analyst Specifically, a modification may be made to the conditional 
upward revision performance score accentuating the number of upward revisions issued by the analyst 



In the above formula, num_up_revl, nmrL_up_rev2, and num_up_rev3 correspond respectively to the 
number of upward revisions for a first interval, the number of upward revisions for a second interval, 
and the number of upward revisions for a third interval. As will be noted from the above formula, three 
separate upward revision numbers or values for three time intervals are considered in this embodiment 



As another example, an upward revision return amount adjustment may be generated to 
emphasize a large or small return amount after a revision forecast made by the analyst This adjustment 
may be utilized to penalize an analyst for a small return, or reward an analyst for a large return. 
Referring to FIG. 9, one example utilizable for determining a return amount adjustment for an upward 

20 revision is illustrated. In this embodiment, the return amount adjustment includes a small return penalty 
for reducing the conditional performance score for returns on the investment that are less than a 
predetermined amount Although any predetermined amount may be utilized, with upward revisions in 
this example, it is set to approximately 15%. The small return penalty is generated using an average of 
a number of return factors and a penalty amount More particularly, the small return penalty is 

25 determined in one embodiment by talcing a difference between an average of a short period of time 

return factor, an intermediate period of time return factor, and a long period of time return factor, and a 
penalty. Thus, each of a short period of time return factor, an intermediate period of time return factor, 
and a long period of time return factor are initially calculated 905, 910, and 915. 



730. 



In this example, the adjustment is determined by: 



up. rev. adj.= (-20)* 




30 



hi this embodiment, subject to certain conditions discussed below, each of the period of time 
return factors (e.g., short, intermediate and long return factors) is equal to a cube root of a return 
percentage over a corresponding period of time, unless the return is greater than approximately 10 or 
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1000%. If the return is greater than approximately 10 or 1000%, the factor is simply set to 10 or 
1000%. Thus to restate the above, if the average return for the period of time is greater than 10, then 
the return factor is simply set to 10. However, if the average return for a period of time is not greater 
than 10, the return factor is equal to: 

5 

return factor = (100 * average upward revision return) 173 

After calculating each of the return values, the optional penalty is generated 920. La this 
embodiment, the penalty is a difference between a predetermined amount and a return for a 

10 predetermined period of time. The predetermined period of time is optionally equal to a time 

corresponding to the long period of time return factor. If the average return value corresponding to the 
investment for the predetermined long period of time is greater than a predetermined amount, the 
penalty is simply set to 0. As an example, the period of time and the predetermined amount may be 
approximately sixty days and approximately 0.15 or 15%, respectively. Thus in this example, a penalty 

15 is calculated only if the return over a sixty day period of time is less than 15%. However, if the average 
return value corresponding to the investment for the long period of time is not greater than 0.15 or 15%, 
the penalty is equal to a product of 100 and a difference between 0.15 (or 15%) and the average return 
value corresponding to the investment for a sixty day period of time. In other words, if the average 
return value corresponding to the investment for the long period of time (e.g., sixty days) is not greater 

20 than 0.15 or 15%: 

penalty = (100 * (0. 15 — average upward revision return for the long period of time)) 

This penalty is then subtracted from the average of die return factors 925 (discussed above) and 
25 used to produce an optional adjustment* 

m 

adjustment = (short return factor + intermediate return factor + long return factor)/3 - penalty 

Thus, in this example, a small return penalty is imposed for small returns (e.g., approximately 
30 15%) over a long period of time (e.g., approximately sixty days) after a revision. Once the return factors 
and penalty have been determined and used to produce an adjustment, the adjustments) is added to the 
conditional performance score to produce a final upward revision performance score 735. 



35 



In the above example, adjustments reflecting the number of revisions issued by the analyst and 
reflecting return amounts emphasizing a small return amount are illustrated. However, as discussed 
above, any other modifications are also possible. For instance, modifications may be made placing a 
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much greater emphasis on short term investments to generate an adjustment aimed at producing scores 
for use in formulating short term investment strategies. Or, in contrast, greater weight may be placed 
on returns over longer periods of time to reflect, for example, a more conservative investment strategy. 

The generation of performance scores based on downward revisions is similar to the generation 
of upward revision performance scores and is depicted in FIG. 8. However, in contrast to upward 
revision performance scores, downward revision performance scores are indicative of an analyst's 
influence or accuracy in predicting the performance of, for example, a security after issuance of a 
downward revision. A conditional downward revision performance score may be manipulated in any 
number of ways to produce refined or final downward revision performance scores. 

As with the upward revision scores, processing commences with the calculation or 
determination of one or more return values, or rates of return, for each analyst, taken a predetermined 
period of time after issuance of the downward revision 80S. Again, any number of return values may 
be taken. In this particular embodiment, three return values, one for a short period of time, one for an 
intermediate period of time, and one for a long period of time, are taken. Furthermore, any lengths may 
be associated with these periods of time (e.g., a short period of time of five days, an intermediate period 
of twenty days, and a long period sixty days). While, three return values are utilized in this 
embodiment, any number of return values, at any number of time intervals, are allowed. 

Subsequently, a statistical measure of confidence in the analyst's performance is calculated 
810. In this case, a t-statistic (t-stat), in absolute trams, is again utilized for exemplary reasons, and is 
likewise calculated for each return value. Again, since three intervals are utilized in this example, three 
t-stat values are calculated, one for each interval. 

For downward revisions, the t-stats for each return period are determined by: 



downward revision return 



downward revision t stat = 



f 



downward revision _ SD 



\ 




Like with the upward revision performance score, to prevent division by zero, if the standard deviations 
for any of the three time intervals are zero, then the downward revision performance score for that 
interval is set to zero. 
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Again, as with the above example, it is to be understood that other measures of confidence may 
be substituted for the t-stat Also, any number of modifications are possible. As indicated above, as 
long as an accurate reflection of the analyst's return history and/or historical consistency at issuing 
downward revisions is considered, any measure of confidence, for example a z-stat or other similar 
5 measurement of variability, dispersion, or consistency may be used. 

After calculating the measures of confidence (e.g., the t-stats for each time interval or return 
period), one or more downward revision degrees of freedom are determined by subtracting, for 
example, one from the number of revisions for each return value 815. Hence, with three return values, 
10 three degrees of freedom are generated Each degree of freedom is determined by: 

downward revision degree of freedom = total number of downward revisions - 1 

Subsequently, a probability indicating, for example, the likelihood that an analyst's and/or other 
IS investing entity's actions will actually produce the expected result, is determined for each return 

interval 820. In this example, the degree of freedom values, along with the absolute t-stat values, are 
used in determining a probability for each return period. Specifically, subject to the following optional 
conditions, the probabilities are looked-up in a common statistics lookup table using each return 
interval's degree of freedom and t-stat If any of the degrees of freedom are greater than 100, then that 
20 degree of freedom is set to 100 before looking up the probability. If any of the absolute t-stats are 

greater than 20, then that absolute t-stat is set to 20 before looking up the probability. Finally, if any of 
the t-stat are less than .01, then the corresponding probability is simply set to 0. Alternatively, the 
probabilities may be calculated based on other the methods. For example, the measures of confidence 
and degrees of freedom may be modified or manipulated before looking up the probability in the 
25 lookup table. Also, any other similar methods may be used in place of the look-up table to generate a 
probability. 

Subsequent to determining the probabilities for each return period, a conditional downward 
revision performance score, or raw conditional downward performance score, is generated 825. 
30 Although other methods are possible, the conditional performance score in this example is determined 

by: 

J t f[l-(.5 + 0r-<to_proM)]+(l-(.5 + or^ K 1f 
cona _dw_ per _ score — < \ 



35 



In this formula, dw_probl, dw_prob2, dw_prob3 correspond respectively to the probability for a first, 
second, and third return interval. Whether the probabilities are added or subtracted is determined by the 



WO 01/80124 



PCIYUS01/U691 



10 



15 



38 

following conditions. In contrast to upward revisions, if any of the three probabilities are less than 
zero, then that probability is subtracted from 0.5. Otherwise, 0.5 is added to the probability. 

As with the above, the conditional downward revision performance score represents an initial 
analyst performance measure and may undergo one or more of any number of manipulations to result in 
various other refined performance measures and ultimately a final downward revision performance 
score. As examples, modifiers accounting for the size of return and the number of revisions issued are 
discussed in greater detail below. These modifications are introduced for exemplary reasons only and it 
is to be understood that other manipulations are also possible within the scope of the present invention. 

For example, a downward revision adjustment may be generated to reflect and emphasize the 
number of revisions issued by the analyst Specifically, a modification may be made to the conditional 
downward revision performance score accentuating the number of downward revisions issued by the 
analyst 



In this example, the adjustment is determined by: 
dw. rev. adj. = (-20)* 



num_dw_revl + num_dw_rev2 + num_dw_rev3 



In the above formula, num_dw_revl, num^dw__rev2, and num^dw_rev3 correspond respectively to the 
number of downward revisions for a first interval, the number of downward revisions for a second 
20 interval, and the number of downward revisions for a third interval. As will be noted from the above 
formula, three separate downward revision numbers or values for three time intervals are considered in 
this embodiment 



As another example, a downward revision return amount adjustment may be generated to 
25 emphasize a large or small return amount after a revision forecast made by the analyst This adjustment 
may be utilized to penalize an analyst for a small return, or reward an analyst for a large return. 
Referring to FIG. 10, one example utilizable for determining a return amount adjustment for a 
downward revision is illustrated. In this embodiment the return amount adjustment includes a small 
return penalty for reducing the conditional performance score for returns on the investment that are less 
30 than a predetermined amount Although any predetermined amount may be utilized, with downward 
revisions, in this example, it is set to approximately negative 8%. The small return penalty is generated 
using an average of a number of return factors and a penalty amount More particularly, the small 
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return penalty is determined by taking a difference between an average of a short period of time return 
factor, an intermediate period of time return factor, and a long period of time return factor, and a 
penalty. Thus, each of a short period of time return factor, an intermediate period of time return factor, 
and a long period of time return factor are initially calculated 1005, 1010, and 1015. 

5 

In this embodiment, subject to certain conditions discussed below, each of the period of time 
return factors (e.g., short, intermediate and long return factors) is equal to a cube root of a return 
percentage over a corresponding period of time, unless the return is greater than approximately 10 or 
1000%. If the return is greater than approximately 10 or 1000%, the factor is simply set to 10 or 
10 1000%. Thus to restate the above, if the average return for the period of time is greater than 10, then 
the return factor is simply set to 10. However, if the average return for a period of time is not greater 
than 10, the return factor is equal to: 

return factor = (100 * average downward revision return) 173 

15 

After calculating each of the return values, the penalty may be generated 1020. In this 
embodiment, the penalty is a difference between a predetermined amount and a return for a 
predetermined period of time. In this example, the predetermined period of time may be equal to a time 
corresponding to the long period of time return factor. If the average return value corresponding to the 

20 investment for the predetermined period of time is less than a predetermined amount, the penalty is 
simply set to 0. As an example, the period of time and the predetermined amount may be 
approximately sixty days and approximately negative 0.08 or -8%, respectively. Thus in this example, 
a penalty is calculated only if the return over a sixty day period of time is greater than negative 8%. 
However, if the average return value corresponding to the investment for the long period of time is not 

25 less than negative 0.08 or -8%, the penalty is equal to a product of 100 and a difference between 
negative 0.08 (or -8%) and the average return value corresponding to the investment for a sixty day 
period of time, hi other words, if the average return value corresponding to the investment for the long 
period of time (e.g., sixty days) is not less than negative 0.08 or -8%: 

30 penalty = (100 * (-0.08 - average downward revision return for the long period of time)) 

This penalty is then subtracted from the average of the return factors 1025 (discussed above) 
and used to produce the adjustment: 

35 adjustment = (short return factor + intermediate return factor + long return factor)/3 - penalty 
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Thus, in this example, a small return penalty is imposed for small returns (e.g., approximately - 
8%) over a long period of time (e.g., approximately sixty days) after a revision. Once the return factors 
and penalty have been determined and used to produce an adjustment, the adjustment(s) are added to 
the conditional performance score to produce a final downward revision performance score 835. 

In accordance with the principles of the present invention, one example of a process for ranking 
analysts according to upward revision performance score is depicted with reference to FIG. 1 1. In 
particular, this process commences by sorting the analysts into descending (or ascending) upward 
revision performance score order 1 10S. Subsequently, as will be discussed in greater detail below, the 
analysts are separated into a number of groups 1 1 10. Each group, then, is assigned a ranking 1 1 IS, 
with, for example, the group with the highest scores receiving a ranking of 100 (signifying the best 
performance) and the group with the lowest scores receiving a ranking of 0 (signifying the worst 
performance). Thus, each analyst receives a rank equal to his or her group. 

Any method or procedure may be utilized to separate or divide the analysts into groups. In one 
embodiment, the analysts are simply divided to form a standard bell curve distribution. In other 
embodiments, other distributions are possible. As one example, the group of analysts is divided into 
seven groups based on upward revision performance score. 

The first of these seven groups includes the top 5% of the analysts as calculated according to 
upward revision performance score. The analysts falling into this first group are then divided into ten 
substantially equally sized subgroups. The highest of these subgroups is assigned a rank of 100. The 
next subgroup receives a rank of 99. This process continues until the last subgroup of the top 5% group 
is assigned a rank of 91. 

The second group includes the next 12.5% of the analysts as calculated according to upward 
revision performance score (e.g., 5% - 12.5%). The analysts falling into the second group are then 
divided into fifteen substantially equally sized subgroups. The highest of these subgroups is assigned a 
rank of 90. The next subgroup receives a rank of 89. This process continues until the last subgroup of 
the second group is assigned a rank of 76. 

The third group includes the next 18% of the analysts as calculated according to upward 
revision performance score (e.g., 12.5% - 18.0%). The analysts falling into the third group are divided 
into fifteen substantially equally sized subgroups. The highest of these subgroups is then assigned a 
rank of 75. The next subgroup receives a rank of 74. This process continues until the last subgroup of 
the second group is assigned a rank of 61. 
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The fourth group includes the next 29% of the analysts as calculated according to upward 
revision performance score (e.g., 18% - 64.5%). The analysts falling into the fourth group are divided 
into twenty substantially equally sized subgroups. The highest of these subgroups is then assigned a 
5 rank of 60. The next subgroup receives a rank of 59. This process continues until the last subgroup of 
the second group is assigned a rank of 41. 

The fifth group includes the next 18% of the analysts as calculated according to upward 
revision performance score (e.g., 64.5% - 82.5%). The analysts falling into the fifth group are then 
10 divided into fifteen substantially equally sized subgroups. The highest of these subgroups is assigned a 
rank of 40. The next subgroup receives a rank of 39. This process continues until the last subgroup of 
the second group is assigned a rank of 26. 

The sixth group includes the next 12.5% of the analysts as calculated according to upward 
15 revision performance score (e.g., 82.5% - 95%). The analysts falling into the sixth group are divided 
into fifteen substantially equally sized subgroups. The highest of these subgroups is then assigned a 
rank of 25. The next subgroup receives a rank of 24. This process continues until the last subgroup of 
the second group is assigned a rank of 11. 

20 The seventh or last group includes the last 5% of the analysts as calculated according to upward 

revision performance score (e.g. f 95% - 100%). The analysts falling into the seventh group are then 
divided into ten substantially equally sized subgroups. The highest of these subgroups is then assigned 
a rank of 10. The next subgroup receives a rank of 9. This process continues until the last subgroup of 
the second group is assigned a rank of 1. 

25 

As depicted in FIG. 12, a similar procedure is utilized to rank the analysts with respect to 
downward revision performance scenes. Thus this process is only briefly summarized here. Initially, 
the process commences by sorting the analysts into descending (or ascending) downward revision 
performance score order 1205. Subsequendy, the analysts are separated into a number of groups 1210. 
30 Then each group is assigned a ranking 1215, with, for example, the group with the highest scores 
receiving a ranking of 100 (signifying the best performance) and the group with the lowest scores 
receiving a ranking of 0 (signifying the worst performance). Like with the upward revision scores, each 
analyst receives a rank equal to his or her group. 

35 Similar to the upward revision procedure used to separate the analysts, any method may be 

utilized to divide these sorted analysts into groups. As one example, the group of analysts is divided 
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into seven groups based on downward revision performance score. For instance, the first group may 
correspond to the top 5% and receive ranks ranging from 100-91. The second group may correspond to 
the next 12.5% of the analysts and receive ranks ranging from 90-76. The third group may correspond 
to the next 18% of the analysts and receive ranks ranging from 75-61. The fourth may correspond to 
5 the next 29% of the analysts and receive ranks ranging from 60- 41. The fifth may correspond to the 
next 18% of the analysts and receive ranks ranging from 40-26. The sixth group may correspond to 
the next 12.5% of the analysts and receive ranks ranging from 25-1 1. The seventh or last group may 
correspond to the last 5% of the analysts and receive ranks ranging from 10-1. 

10 In addition, as mentioned above, any other similar and analogous methods may be used to 

separate and/or divide the analysts into groups. For instance, each group may be required to include an 
equal number of analysts. Similarly, different criteria may be used to determine the groups. 

One example depicting the tabulated results of processing in accordance with the above 
15 techniques with respect to a group of analysts covering a single company (IBM) is shown in FIG. 18. 
In this example, both upward and downward performance scores are listed. Also, as discussed above, 
the scores may be averaged or otherwise combined to yield a combined performance score. Other 
alternatives are also possible. 

20 Viewed externally in FIG. 13, a computer system designated by reference numeral 140 has a 

computer 142 having disk drives 144 and 146. Disk drive indications 144 and 146 are merely symbolic 
of a number of disk drives which might be accommodated by the computer system. Typically, these 
would include a floppy disk drive 144, a hard disk drive (not shown externally) and a CD ROM 
indicated by slot 146. The number and type of drives vary, typically with different computer 

25 configurations. Disk drives 144 and 146 are in fact optional, and for space considerations, are easily 
omitted from the computer system used in conjunction with the production process/apparatus described 
herein. 

The computer system also has an optional display upon which information, such as the screen 
30 illustrated in FIG. 5 may be displayed. In some situations, a keyboard 150 and a mouse 152 are 
provided as input devices through which a user's actions may be inputted, thus allowing input to 
interface with the central processing unit 142. Then again, for enhanced portability, the keyboard 150 
is either a limited function keyboard or omitted in its entirety. In addition, mouse 152 optionally is a 
touch pad control device, or a track ball device, or even omitted in its entirety as well, and similarly 
35 may be used toinput a user's selections. In addition, the computer system also optionally includes at 
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least one infrared transmitter and/or infrared received for either transmitting and/or receiving infrared 
signals, as described below. 

FIG. 14 illustrates a block diagram of the internal hardware of the computer system 140 of FIG. 
5 13. A bus 156 serves as the main information highway interconnecting the other components of the 
computer system 140. CPU 158 is the central processing unit of the system, performing calculations 
and logic operations required to execute the processes of the instant invention as well as other 
programs. Read only memory (ROM) 160 and random access memory (RAM) 162 constitute the main 
memory of the computer. Disk controller 164 interfaces one or more disk drives to the system bus 156. 
10 These disk drives are, for example, floppy disk drives such as 170, or CD ROM or DVD (digital video 
disks) drive such as 166, or internal or external hard drives 168. As indicated previously, these various 
disk drives and disk controllers are optional devices. 

A display interface 172 interfaces display 148 and permits information from the bus 156 to be 

15 displayed on the display 148. Again as indicated, display 148 is also an optional accessory. For 
example, display 148 could be substituted or omitted. Communications with external devices, for 
example, the other components of the system described herein, occur utilizing communication port 174. 
For example, optical fibers and/or electrical cables and/or conductors and/or optical communication 
(e.g., infrared, and the like) and/or wireless communication (e.g., radio frequency (RF), and the like) 

20 can be used as the transport medium between the external devices and communication port 174. 
Peripheral interface 154 interfaces the keyboard 150 and the mouse 152, permitting input data to be 
transmitted to the bus 156. In addition to the standard components of the computer, the computer also 
optionally includes an infrared transmitter 178 and/or infrared receiver 176. Infrared transmitters are 
optionally utilized when the computer system is used in conjunction with one or more of the processing 

25 components/stations that transmits/receives data via infrared signal transmission. Instead of utilizing an 
infrared transmitter or infrared receiver, the computer system may also optionally use a low power 
radio transmitter 180 and/or a low power radio receiver 182 as shown in the alternate embodiment of 
FIG. 15. The low power radio transmitter transmits the signal for reception by components of the 
production process, and receives signals from the components via the low power radio receiver. The 

30 low power radio transmitter and/or receiver are standard devices in industry. 

FIGS. 16 is an illustration of an exemplary memory medium 184 which can be used with disk 
drives illustrated in FIGS. 14-15. Typically, memory media such as floppy disks, or a CD ROM, or a 
digital video disk will contain, for example, a multi-byte locale for a single byte language and the 
35 program information for controlling the computer to enable the computer to perform the functions 

described herein. Alternatively, ROM 160 and/or RAM 162 illustrated in FIGS. 14-15 can also be used 
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to store the program information that is used to instruct the central processing unit 158 to perform the 
operations associated with the instant processes. 

Although computer system 140 is illustrated having a single processor, a single hard disk drive 
5 and a single local memory, the system 140 is optionally suitably equipped with any multitude or 
combination of processors or storage devices. Computer system 140 is, in point of fact, able to be 
replaced by, or combined with, any suitable processing system operative in accordance with the 
principles of the present invention, including sophisticated calculators, and hand-held, laptop/notebook, 
mini, mainframe and super computers, as well as processing system network combinations of the same. 

10 

Conventional processing system architecture is more fully discussed in Computer Organization 
and Architecture , by William Stallings, MacMillan Publishing Co. (3rd ed. 1993); conventional 
processing system network design is more fully discussed in Data Network Design , by Darren L. 
Spohn, McGraw-Hill, Inc. (1993), and conventional data communications are more fully discussed in 

15 Data Comn nmicarinTi s Principles , by R.D. Gitlin, LF. Hayes and S.B. Weinstain, Plenum Press (1992) 
and in The Irwin Handbook of Telecommunications , by James Harry Green, Irwin Professional 
Publishing (2nd ed. 1992). Each of the foregoing publications is incorporated herein by reference. 
Alternatively, the hardware configuration is, for example, arranged according to the multiple instruction 
multiple data (MIMD) multiprocessor format for additional computing efficiency. The details of this 

20 form of computer architecture are disclosed in greater detail in, for example, U.S. Patent No. 5,163,13 1; 
Boxer, A., Where Buses Cannot Go, ihhk Spectrum, February 1995, pp. 41-45; and Barroso, L.A. et 
al., RPM: A Rapid Prototyping Engine for Multiprocessor Systems, IEEE Computer February 1995, pp. 
26-34, all of which are incorporated herein by reference. 

25 In alternate preferred embodiments, the above-identified processor, and, in particular, CPU 

158, may be replaced by or combined with any other suitable processing circuits, including 
programmable logic devices, such as PALs (programmable array logic) and PLAs (programmable logic 
arrays). DSPs (digital signal processors), FPGAs (field programmable gate arrays), ASICs (application 
specific integrated circuits), VLSIs (very large scale integrated circuits) or the like. 

30 

FIG. 17 is an illustration of the architecture of the combined Internet, POTS (plain, old, 
telephone service), and ADSL (asymmetric, digital, subscriber line) for use in accordance with the 
principles of the present invention. Furthermore, it is to be understood that the use of the Internet, 
ADSL, and POTS are for exemplary reasons only and that any suitable communications network may 
35 be substituted without departing from the principles of the present invention. This particular example is 
briefly discussed below. 
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In FIG. 17, to preserve POTS and to prevent a fault in the ADSL equipment 254, 256 from 
compromising analog voice traffic 226, 296 the voice part of the spectrum (the lowest 4 kHz) is 
separated from the rest by a passive filter, called a POTS splitter 258, 260. The rest of the available 
5 bandwidth - - from about 10 kHz to 1 MHz - - carries data at rates up to 6 bits per second for every 
hertz of bandwidth from data equipment 262, 264, and 294. The ADSL equipment 256 then has access 
to a number of destinations including significantly the Internet 220 or other data communications 
networks, and other destinations 270, 272. 

10 To exploit the higher frequencies, ADSL makes use of advanced modulation techniques, of 

which the best known is the discrete multitone (DMT) technology. As its name implies, ADSL 
transmits data asymmetrically - - at different rates upstream toward the central office' 252 and 
downstream toward the subscriber 250. 

15 Cable television providers are providing analogous Internet service to PC users over their TV 

cable systems by means of special cable modems. Such modems are capable of transmitting up to 30 
Mb/s over hybrid fiber/coax system, which use fiber to bring signals to a neighborhood and coax to 
distribute it to individual subscribers. 

20 Cable modems come in many forms. Most create a downstream data stream out of one of the 

6-MHz TV channels that occupy spectrum above 50 MHz (and more likely 550 MHz) and carve an 
upstream channel out of the 5-50-MHz band, which is currently unused. Using 64-state quadrature 
amplitude modulation (64 QAM), a downstream channel can realistically transmit about 30 Mb/s (the 
oft-quoted lower speed of 10 Mb/s refers to PC rates associated with Ethernet connections). Upstream 

25 rates differ considerably from vendor to vendor, but good hybrid fiber/coax systems can deliver 

upstream speeds of a few megabits per second. Thus, like ADSL, cable modems transmit much more 
information downstream than upstream. Then Internet architecture 220 and ADSL architecture 254, 
256 may also be combined with, for example, user networks 222, 224, and 228. 

30 In accordance with the principles of the present invention, in one example, a main computing 

server implementing the process of the invention may be located on one computing node or terminal 
(e.g., on user network 222, or system 240). Then, various users may interface with the main server via, 
for instance, the ADSL equipment discussed above, and access the information and processes of the 
present invention from remotely located PCs. More specifically, a process 242 capable of performing 

35 the analyst evaluating procedure of the instant invention may be advantageously implemented in system 
240. As depicted in FIG. 17, process 242 may access a list of analysts stored, for instance in data 
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structure 294. Furthermore, although data structure 294 is shown as being stored in system 240, it may 
just as easily be located in a remote and distinct computer system. For example, process 242 may be 
implemented in such a manner as to have access to one of any number of databases 281, 282, 283, 285, 
286, 287 or 288. 

Furthermore, the analyst evaluating process of the present invention may also be implemented 
manually. For instance, it is possible to evaluate analysts by hand without the assistance of computing 
systems. 

The many features and advantages of the invention are apparent from the detailed specification, 
and thus, it is intended by the appended claims to cover all such features and advantages of the 
invention which fall within the true spirit and scope of the invention. Further, since numerous 
modifications and variations will readily occur to those skilled in the art, it is not desired to limit the 
invention to the exact construction and operation illustrated and described, and accordingly, all suitable 
modifications and equivalents may be resorted to, falling within the scope of the invention. 
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CLAIMS 

What is claimed is: 

SI. A method of evaluating an analyst's performance, said method comprising the steps of: 

utilizing information pertaining to at least one revision made by the analyst involving at least 
one investment, including an upward or downward change in opinion of the analyst with regard to the 
at least one investment; 

calculating a conditional performance score indicative of the analyst's performance relative to 
10 other analysts, said raw conditional performance score determined at least in part by considering a 
measure of variability of the analyst's performance, an average historical performance of the at least 
one investment following the at least one revision, a standard deviation of the at least one revision, a 
number of revisions made by the analyst, and a likelihood that the at least one revision will actually 
produce an expected result; and 
15 adjusting said conditional performance score according to a return amount adjustment to 

produce a final performance score, 

2. The method of claim 1, wherein the average historical performance of the at least one 
investment is considered over three distinct periods of time. 

20 

3. The method of claim 2, wherein said three distinct periods of time comprise a short period of 
time which is approximately five days, an intermediate period of time which is approximately twenty 
days, and a long period of time which is approximately sixty days. 

25 4. The method of claim 1, wherein the final performance score increases with the number of 

revisions. 

5. The method of claim 1, wherein said revision comprises one of an upward revision or a 
downward revision. 

30 

6. The method of claim 1, wherein the final performance score increases when a rate of return of 
the at least one investment increases and the revision is an upward revision, or when a rate of return of 
the at least one investment decreases and the revision is a downward revision. 
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7. The method of claim 1, wherein the final performance score decreases when a rate of return of 
the at least one investment decreases and the revision is an upward revision, or when a rate of return of 
the at least one investment increases and the revision is a downward revision. 

S 8. The method of claim 1, further comprising adjusting said conditional performance score 
according to a modifier accentuating the number of revisions made by the analyst 

9. The method of claim 1 , wherein said return amount adjustment comprises a small return 
penalty reducing said final performance score for returns on said at least one investment less than a 

10 predetermined amount. 

10. The method of claim 9, wherein said small return penalty is comprised of a difference between 
an average of a short period of time return factor, an intermediate period of time return factor, and a 
long period of time return factor, and a penalty; wherein said penalty is equal to approximately 15% for 

IS upward revisions and approximately -8% for downward revisions minus a return value corresponding 
to the investment for said long period of time; and wherein each period of time return factor is a cube 
root of a return percentage over a corresponding period of time. 

11. The method of claim 9, wherein said small return penalty is imposed for small returns over a 
20 long period of time of approximately sixty days after said at least one revision. 

12. The method of claim 9, wherein said small return penalty is generated by subtracting a penalty 
from an average of a first, second and third period of time return factor, wherein for upward revisions 
said penalty is 0 if an average return value corresponding to the investment for a long term period of 

25 time is greater than 15%, else said penalty is equal to 15% minus said average return value 

corresponding to the investment for a long term period of time; wherein for downward revisions said 
penalty is 0 if said average return value corresponding to the investment for a long term period of time 
is less than -8%, else said penalty is equal to -8% minus said average return value corresponding to the 
investment for a long term period of time; wherein said first period of time return factor is ten if an 

30 average return over said first period of time is greater than ten, else said first period of time return 

factor is equal to a cube root of a product of 100 and said average return over said first period of time; 
wherein said second period of time return factor is ten if an average return over said second period of 
time is greater than ten, else said second period of time return factor is equal to a cube root of a product 
of 100 and said average return over said second period of time; and wherein said third period of time 

35 return factor is ten if an average return over said third period of time is greater than ten, else said third 
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period of time return factor is equal to a cube root of a product of 100 and said average return over said 
third period of time. 

13. The method of claim 1, wherein said calculating a conditional performance score comprises: 
5 determining a first average return value (ave. returnO, second average return value (ave. 

retun^), and third average return value (ave. returns) for the at least one revision, said first, second, and 
third average return values corresponding respectively to an average rate of return on the at least one 
revision involving the investment for a first period of time (timeO, a second period of time (timez), and 
a third period of time (times); 

10 calculating, in absolute terms, a first t-stat value (t-stati) for said first average return value, a 

second t-stat value (t-stai2) for said second average return value, and a third t-stat value (t-stat 3 ) for said 
third average return value, said first, second, and third t-stat values calculated by utilizing said first, 
second, and third average return values, a first standard deviation (SDj), a second standard deviation 
(SD2), and a third standard deviation (SD3), a total number of revisions (revisions^ occurring more than 

15 or equal to the first period of time before said evaluating, a total number of revisions (revisions^ 

occurring more than or equal to the second period of time before said evaluating, and a total number of 
revisions (revisionsa) occurring more than or equal to the third period of time before said evaluating; 

calculating a first degree of freedom (DFi) for said first average return value, a second degree 
of freedom (DF2) for said second average return value, and a third degree of freedom (DF 3 ) for said 

20 third average return value; 

determining a first probability (probi) for said first period of time by utilizing said first t-stat 
value and said first degree of freedom, a second probability (prob^ for said second period of time by 
utilizing said second t-stat value and said second degree of freedom, and a third probability (prol^) for 
said third period of time by utilizing said third t-stat value and said third degree of freedom; 

25 generating a raw conditional performance score for the analyst by utilizing probt , prob^ and 

prob3; 

generating an adjustment to said raw conditional performance score by utilizing revisions lf 
revisions2, and revisionss; and 

adding said adjustment to said raw conditional performance score to produce said conditional 
30 performance score. 



14. The method of claim 13, wherein if SDi = 0, then conditional performance score = 0 otherwise 



WO 01/80124 



PCT/US01/11691 



50 



Tstatt = 



ave. return! 



SD1 } 



previsions! 



; wherein if SD2, = 0, then conditional performance score = 0 otherwise 

ave. return! 



Tstatt 



SD2 



*Jrevisions2 



10 



; wherein if SD3, = 0, then conditional performance score = 0 otherwise 

ave. return3 



Tstati - 



SD3 



V revisions^ 



; wherein DFi = revisions! - 1; DF 2 = revisions2 - 1; and DF 3 = revisions 3 - 1 

15. The method of claim 13, wherein said probabilities are determined according to a 
predetermined look-up table using actual calculated values of: DFi, DF 2 , DF 3t t-stati, t-stat 2 , and t-stat 3 , 
unless: DFi > 100, in which case DFi is set to 100; DF 2 > 100, in which case DF 2 is set to 100; DF 3 > 
100, in which case DF 3 is set to 100; t-stati > 20, in which case t-stati is set to 20; t-stat 2 > 20, in which 
case t-stat 2 is set to 20; t-stat 3 > 20, in which case t-stat 3 is set to 20; t-stati < 0.01, in which case probi is 
set to 0; t-stat 2 < 0.01, in which case prob^ is set to 0; or t~stat 3 < 0.01 , in which case prol^ is set to 0. 



15 16. The method of claim 13, wherein 



cond _ performance _ score 



= |[l-(^+or-pro61)] + [\-(5+or- prob2)]+[l-(5+or- probi)] j # 



100 



wherein for upward revisions probi is subtracted from 0.5 if ave. returni > 0, else probi is added to 0.5 
wherein for upward revisions prot^ is subtracted from 0.5 if ave. return 2 > 0, else prot^ is added to 0.5 
wherein for upward revisions probs is subtracted from 0.5 if ave. return 3 > 0, else prot^ is added to 0.5 
20 wherein for downward revisions probi is subtracted from 0.5 if ave. returni < 0, else probi is added to 
0.5; wherein for downward revisions prol^ is subtracted from 0.5 if ave. return 2 < 0, else prolh is added 
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to 0.5; and wherein for downward revisions prob* is subtracted from 0.5 if ave. return 3 < 0, else probj is 
added to 0.5. 

17. The method of claim 13, wherein 



18. A method of evaluating an analyst's performance, said method comprising the steps of: 
utilizing information pertaining to at least one upward or downward revision including an 

upward or downward change in opinion of the analyst with regard to the at least one investment; 
calculating a performance score indicative of the analyst's performance relative to other 

10 analysts, said performance score determined at least in part by considering a measure of variability of 
the analyst's performance, an average historical performance of the at least one investment following 
the at least one upward or downward revision, a standard deviation of the at least one upward or 
downward revision, a number of upward or downward revisions made by the analyst, and a likelihood 
that the at least one upward or downward revision will actually produce an expected result; and 

15 comparing the analyst's performance score against performance scores of other analysts to 

produce at least one of an upward, downward or combined revision ranking. 

19. The method of claim 18, wherein said three distinct periods of time comprise a short period of 
time which is approximately five days, an intermediate period of time which is approximately twenty 

20 days, and a long period of time which is approximately sixty days. 

20. The method of claim 18, wherein said performance score increases with the number of 
revisions. 

25 21. The method of claim 18, further comprising adjusting said performance score according to a 
modifier accentuating the number of revisions made by the analyst 

22. The method of claim 18, wherein said performance score increases when a rate of return of the 
at least one investment increases and the revision is an upward revision, or when a rate of return of the 
30 at least one investment decreases and the revision is a downward revision, and wherein said 

performance score decreases when a rate of return of the at least one investment decreases and the 
revision is an upward revision, or when a rate of return of the at least one investment increases and the 
revision is a downward revision. 



adjustment = (-20) * 




5 
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23. The method of claim 18, further comprising adjusting said performance score according to a 
small return penalty. 

S 24. The method of claim 23, wherein said small return penalty is generated by subtracting a penalty 
from an average of a first, second and third period of time return factor, wherein for upward revisions 
said penalty is 0 if an average return value corresponding to the investment for a long term period of 
time is greater than 15%, else said penalty is equal to 15% minus said average return value 
corresponding to the investment for a long term period of time; wherein for downward revisions said 

10 penalty is 0 if said average return value corresponding to the investment for a long term period of time 
is less than -8%, else said penalty is equal to -8% minus said average return value corresponding to the 
investnent for a long term period of time; wherein said first period of time return factor is ten if an 
average return over said first period of time is greater than ten, else said first period of time return 
factor is equal to a cube root of a product of 100 and said average return over said first period of time; 

15 wherein said second period of time return factor is ten if an average return over said second period of 
time is greater than ten, else said second period of time return factor is equal to a cube root of a product 
of 100 and said average return over said second period of time; and wherein said third period of time 
return factor is ten if an average return over said third period of time is greater than ten, else said third 
period of time return factor is equal to a cube root of a product of 100 and said average return over said 

20 third period of time. 

25. The method of claim 18, wherein said calculating a performance score comprises: 

determining a first average return value (ave. returni), second average return value (ave. 
return^, and third average return value (ave. retum 3 ) for the at least one revision, said first, second, and 

25 third average return values corresponding respectively to an average rate of return on the at least one 
revision involving the investment for a first period of time (timet), a second period of time (timez), and 
a third period of time (times); 

calculating, in absolute terms, a first t-stat value (t-stati) for said first average return value, a 
second t-stat value (t-stat2) for said second average return value, and a third t-stat value (t-stat 3 ) for said 

30 third average return value, said first, second, and third t-stat values calculated by utilizing said first, 
second, and third average return values, a first standard deviation (SDJ, a second standard deviation 
(SDJ, and a third standard deviation (SD3), a total number of revisions (revisionsO occurring more than 
or equal to the first period of time before said evaluating, a total number of revisions (revisions^ 
occurring more than or equal to the second period of time before said evaluating, and a total number of 

35 revisions (revisions3) occurring more than or equal to the third period of time before said evaluating; 
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calculating a first degree of freedom (DF{) for said first average return value, a second degree 
of freedom (DF2) for said second average return value, and a third degree of freedom (DF 3 ) for said 
third average return value; 

determining a first probability (probi) for said first period of time by utilizing said first t-stat 
value and said first degree of freedom, a second probability (prob^ for said second period of time by 
utilizing said second t-stat value and said second degree of freedom, and a third probability (probj) for 
said third period of time by utilizing said third t-stat value and said third degree of freedom; and 

generating a conditional performance score (cond. performance score) for the analyst by 
utilizing probj, prot>2, and prot>3 t , which may be manipulated to produce said performance score. 

26. The method of claim 25, wherein if SDj = 0, then conditional performance score = 0 otherwise 

ave. returnl 



T stat \ 



SDl 



V revisions! 



; wherein if SD2, = 0, then conditional performance score = 0 otherwise 

ave. returnl 



Tstat 2 = 



SD2 



previsions! 



15 ; wherein if SD3, = 0, then conditional performance score = 0 otherwise 

ave. return3 



T stats ~ 



SD3 



V revisions^ 



; wherein DFi - revisions! - 1; DF 2 = revisionsa - 1; and DF 3 = revisions 3 - 1 



27. The method of claim 25, wherein said probabilities are determined according to a 
20 predetermined look-up table using actual calculated values of: DFi, DF 2 , DF 3 , t-stati, t-stat2» and t-stat 3 , 
unless: DFi > 100, in which case DF t is set to 100; DF 2 > 100, in which case DF 2 is set to 100; DF 3 > 
100, in which case DF 3 is set to 100; t-stati > 20, in which case t-stati is set to 20; t-stat 2 > 20, in which 
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case t-stai 2 is sec to 20; t-stat 3 >20, in which case t-stat 3 is set to 20; t-stati < 0.01, in which case probi is 
set to 0; t-stat 2 < 0.01, in which case proth is set to 0; or t-stat 3 < 0.01, in which case prob3 is set to 0. 



28. The method of claim 25, wherein 

. , f [l-C5+or-/>roM)]+[l-(^ 1 
5 cond _ performance _ score = j f 10 °, 

wherein for upward revisions prob! is subtracted from 0.5 if ave. retunii > 0, else prob 1 is added to 0.5; 
wherein for upward revisions prot>2 is subtracted from 0.5 if ave. return 2 > 0, else prot>2 is added to 0.5; 
wherein for upward revisions prot>3 is subtracted from 0.5 if ave. return 3 > 0, else prob3 is added to 0.5; 
wherein for downward revisions probi is subtracted from 0.5 if ave. returni < 0, else probi is added to 
10 0.5; wherein for downward revisions prot>2 is subtracted from 0.5 if ave. return < 0, else prot>2 is added 
to 0.5; and wherein for downward revisions prot^ is subtracted from 0.5 if ave. return 3 < 0, else prot>3 is 
added to 0.5. 



29. A method of evaluating an analyst's performance by utilizing information pertaining to at least 
15 one revision made by the analyst involving at least one investment, said method comprising the steps 
of: 

determining a first average return value (ave. returni), second average return value (ave. 
return^, and third average return value (ave. returns) for the at least one revision, said first, second, and 
third average return values corresponding respectively to an average rate of return on the at least one 
20 revision involving the investment for a first period of time (timeO, a second period of time (time^, and 
a third period of time (tim^); 

calculating, in absolute terms, a first t-stat value (t-stati) for ave. return!, a second t-stat value 
(t-stat2) for ave. return 2 , and a third t-stat value (t-stat 3 ) for ave. retum 3 , said first, second, and third t- 
stat values calculated by utilizing ave. returni, ave. return^ ave. returns, a first standard deviation (SDi), 
25 a second standard deviation (SD2), and a third standard deviation (SD3), a total number of revisions 
(revisions 1) occurring more than or equal to timet before said evaluating, a total number of revisions 
(revisions^ occurring more than or 6qual to time* before said evaluating, and a total number of 
revisions (revisians 3 ) occurring more than or equal to time 3 before said evaluating; 

calculating a first degree of freedom (DFi) for ave. return^ a second degree of freedom (DF2) 
30 for ave. return* and a third degree of freedom (DF 3 ) for ave. return 3 ; 

determining a first probability (probi) for timet by utilizing t-stati and DF 1? a second probability 
(probz) for time2 by utilizing t-stat 2 and DF 2 , and a third probability (probs) for time 3 by utilizing t-stat 3 
and DF 3 ; 

generating a raw conditional performance score for the analyst by utilizing probi, prol^, prot^; 
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adding a modifier (adjustment) to the raw conditional performance score determined by 
utilizing revisions!, revisions*, and revisions 3 , thereby producing a conditional performance score 
(cond. performance score); . 

generating a small return penalty by subtracting a penalty from an average of return factors 
5 corresponding to timei, time^ and time^ wherein for upward revisions said penalty is 0 if ave. return 3 is 
greater than 15%, else said penalty is equal to 15% minus ave. retum 3 , wherein for downward revisions 
said penalty is 0 if ave. return 3 is less than -8%, else said penalty is equal to -8% minus ave. retum 3 , 
wherein said first period of time return factor is ten if ave. return! is greater than ten, else said first 
period of time return factor is equal to a cube root of a product of 100 and ave. returni, wherein said 
10 second period of time return factor is ten if ave. return 2 is greater than ten, else said second period of 
time return factor is equal to a cube root of a product of 100 and ave. return*, and wherein said third 
period of time return factor is ten if ave. return 3 is greater than ten, else said third period of time return 
factor is equal to a cube root of a product of 100 and ave. retum 3 ; and 

generating a final performance score (performance score) indicative of said analyst's 

15 performance by adding said small return penalty to said conditional performance score. 

j 

30. A system for evaluating an analyst's performance, said system comprising: 
a processor; 

a memory storing a computer program controlling operation of said processor, said program 
20 including instructions for causing the processor to effect 

utilizing information pertaining to at least one revision made by the analyst involving at least 
one investment, including an upward or downward change in opinion of the analyst with regard to the 
at least one investment; 

calculating a conditional performance score indicative of the analyst's performance relative to 
25 other analysts, said raw conditional performance score determined at least in part by considering a 
measure of variability of the analyst's performance, an average historical performance of the at least 
one investment following the at least one revision, a standard deviation of the at least one revision, a 

number of revisions made by the analyst, and a likelihood that the at least one revision will actually 

* • 

produce an expected result; and 
30 adjusting said conditional performance score according to a return amount adjustment to 

produce a final performance score. 

3 1 . The system of claim 30, wherein said average historical performance is calculated over three 
distinct periods of time, which comprise a short period of time which is approximately five days, an 

35 intermediate period of time which is approximately twenty days, and a long period of time which is 
approximately sixty days. 
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32. The system of claim 30, wherein the final performance score increases when a rate of return of 
the at least one investment increases and the revision is an upward revision, or when a rate of return of 
the at least one investment decreases and the revision is a downward revision, and wherein the final 

5 performance score decreases when a rate of return of the at least one investment decreases and the 
revision is an upward revision, or when a rate of return of the at least one investment increases and the 
revision is a downward revision. 

33. The system of claim 30, wherein said program further comprises instructions to effect adjusting 
10 said conditional performance score according to a modifier accentuating the number of revisions made 

by the analyst 

34. The system of claim 30, wherein said return amount adjustment comprises a small return 
penalty reducing said final performance score for returns on said at least one investment less than a 

IS predetermined amount 

35. The system of claim 34, wherein said small return penalty is comprised of a difference between 
an average of a short period of time return factor, an intermediate period of time return factor, and a 
long period of time return factor, and a penalty; wherein said penalty is equal to approximately 15% for 

20 upward revisions and approximately -8% for downward revisions minus a return value corresponding 
to the investment for said long period of time; and wherein each period of time return factor is a cube 
root of a return percentage over a corresponding period of time. 

36. The system of claim 34, wherein said small return penalty is imposed for small returns over a 
25 long period of time of approximately sixty days after said at least one revision. 

37. The system of claim 34, wherein said small return penalty is generated by subtracting a penalty 
from an average of a first, second and third period of time return factor; wherein for upward revisions 
said penalty is 0 if an average return value corresponding to the investment for a long term period of 

30 time is greater than 15%, else said penalty is equal to 15% minus said average return value 

corresponding to the investment for a long term period of time; wherein for downward revisions said 
penalty is 0 if said average return value corresponding to the investment for a long term period of time 
is less than -8%, else said penalty is equal to -8% minus said average return value corresponding to the 
investment for a long term period of time; wherein said first period of time return factor is ten if an 

35 average return over said first period of time is greater than ten, else said first period of time return 

factor is equal to a cube root of a product of 100 and said average return over said first period of time; 



WO 01/80124 PCT/US01/1 1691 

57 

wherein said second period of time return factor is ten if an average return over said second period of 
time is greater than ten, else said second period of time return factor is equal to a cube root of a product 
of 100 and said average return over said second period of time; and wherein said third period of time 
return factor is ten if an average return over said third period of time is greater than ten t else said third 
5 period of time return factor is equal to a cube root of a product of 100 and said average return over said 
third period of time. 

3. A system for evaluating an analyst's performance, said system comprising: 

a processor, 

10 a memory storing a computer program controlling operation of said processor, 
said program including instructions for causing the processor to effect 

utilizing information pertaining to at least one upward or downward revision including an 
upward or downward change in opinion of the analyst with regard to the at least one investment; 
calculating a performance score indicative of the analyst's performance relative to other 

IS analysts, said performance score determined at least in part by considering a measure of variability of 
the analyst's performance, an average historical performance of the at least one investment following 
the at least one upward or downward revision, a standard deviation of the at least one upward or 
downward revision, a number of upward or downward revisions made by the analyst, and a likelihood 
that the at least one upward or downward revision will actually produce an expected result; and 

20 comparing the analyst's performance score against performance scores of other analysts to 

produce at least one of an upward, downward or combined revision ranking. 

39. The system of claim 38, wherein said program further comprises instructions to effect adjusting 
said performance score according to a modifier accentuating the number of revisions made by the 

25 analyst 

40. The system of claim 38, wherein said performance score increases when a rate of return of the 
at least one investment increases and the revision is an upward revision, or when a rate of return of the 
at least one investment decreases and the revision is a downward revision, and wherein said 

30 performance score decreases when a rate of return of the at least one investment decreases and the 

revision is an upward revision, or when a rate of return of the at least one investment increases and the 
revision is a downward revision. 

4 1 . The system of claim 38, wherein said calculating a performance score comprises: 

35 determining a first average return value (ave. returni), second average return value (ave. 

returns), and third average return value (ave. returns) for the at least one revision, said first, second, and 
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third average return values corresponding respectively to an average rate of return on the at least one 
revision involving the investment for a first period of time (timeO, a second period of time (time^ and 
a third period of time (tune 3 ); 

calculating, in absolute terms, a first t-stat value (t-stat t ) for said first average return value, a 
second t-stat value (t-stat^) for said second average return value, and a third t-stat value (t-stat 3 ) for said 
third average return value, said first, second, and third t-stat values calculated by utilizing said first, 
second, and third average return values, a first standard deviation (SD0, a second standard deviation 
(SD2), and a third standard deviation (SIX,), a total number of revisions (revisions 1) occurring more than 
or equal to the first period of time before said evaluating, a total number of revisions (revisions^ 
occurring more than or equal to the second period of time before said evaluating, and a total number of 
revisions (revisions 3 ) occurring more than or equal to the third period of time before said evaluating; 

calculating a first degree of freedom (DFO for said first average return value, a second degree 
of freedom (DF2) for said second average return value, and a third degree of freedom (DF 3 ) for said 
third average return value; 

determining a first probability (probi) for said first period of time by utilizing said first t-stat 
value and said first degree of freedom, a second probability (prob^ for said second period of time by 
u tilisin g said second t-stat value and said second degree of freedom, and a third probability (prob>3) for 
said third period of time by utilizing said third t-stat value and said third degree of freedom; and 

generating a conditional performance score (cond. performance score) for the analyst by 
utilizing probi, prot>2, and prob^, which may be manipulated to produce said performance score. 

42. The system of claim 41, wherein if SDj = 0, then conditional performance score = 0 otherwise 



aye. returnl 



Tstatx = 




; wherein if SD2, = 0, then conditional performance score = 0 otherwise 

ave, returnl 



Tstat 2 = 



SD2 



V revisions! 



J 



; wherein if SD3, = 0, then conditional performance score = 0 otherwise 

I ave. return3 
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Tstatx = 



10 



f SD3 ^ 
^Jrevisions3 j 

; wherein DF X = revisions t - 1 ; DF 2 = revision - 1; and DF 3 = revisions3 - 1 

43. The system of claim 41, wherein said probabilities are determined according to a predetermined 
look-up table using actual calculated values of: DF U DF 2 , DF 3 , t-statj, t-stat 2 , and t-stat 3 , unless; DF ( > 
100, in which case DFi is set to 100; DF 2 > 100, in which case DF 2 is set to 100; DF 3 > 100, in which 
case DF 3 is set to 100; t-stati > 20, in which case t-stati is set to 20; t-stat 2 > 20, in which case t-stat 2 is 
set to 20; t-stat 3 > 20, in which case t-stat 3 is set to 20; t-stat t <0.01, in which case probi is set to 0; t- 
stat 2 < 0.01, in which case prot^ is set to 0; or t-stat 3 < 0.01, in which case prot^ is set to 0. 

44. The system of claim 41, wherein 



cond _ performance _ score 



_ |[l - (.5 + or - probl)} + [l - (.5 + or - prob2)]+ [l - (5 + or - prob3)] ^^100 



wherein for upward revisions probt is subtracted from 0.5 if ave. return i > 0, else probi is added to 0.5 
wherein for upward revisions pro^ is subtracted from 0.5 if ave. return 2 > 0, else prol^ is added to 0.5 
15 wherein for upward revisions probj is subtracted from 0.5 if ave. return 3 > 0, else prob) is added to 0.5 
wherein for downward revisions probi is subtracted from 0.5 if ave. returni < 0, else probi is added to 
0.5; wherein for downward revisions prot^ is subtracted from 0.5 if ave. retun^ < 0, else prot^ is added 
to 0.5; and wherein for downward revisions pro^ is subtracted from 0.5 if ave. return 3 < 0, else probj is 
added to 0.5. 

20 

45. A system for evaluating an analyst's performance, said system comprising: 

means for utilizing information pertaining to at least one revision made by the analyst involving 
at least one investment, including an upward or downward change in opinion of the analyst with regard 
to the at least one investment; 

25 means for calculating a conditional performance score indicative of the analyst' s performance 

relative to other analysts, said raw conditional performance score determined at least in part by 
considering a measure of variability of the analyst's performance, an average historical performance of 
the at least one investment following the at least one revision, a standard deviation of the at least one 
revision, a number of revisions made by the analyst, and a likelihood that the at least one revision will 

30 actually produce an expected result; and 
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means for adjusting said conditional performance score according to a return amount 
adjustment to produce a final performance score. 

46. The system of claim 45, wherein said average historical performance is calculated over three 
5 distinct periods of time, which comprise a short period of time which is approximately five days, an 

intermediate period of time which is approximately twenty days, and a long period of time which is 
approximately sixty days. 

47. The system of claim 45, wherein the final performance score increases when a rate of return of 
10 the at least one investment increases and the revision is an upward revision, or when a rate of return of 

the at least one investment decreases and the revision is a downward revision, and wherein the final 
performance score decreases when a rate of return of the at least one investment decreases and the 
revision is an upward revision, or when a rate of return of the at least one investment increases and the 
revision is a downward revision. 

15 

48. The system of claim 45, further comprising means for adjusting said conditional performance 
score according to a modifier accentuating the number of revisions made by the analyst 

49. The system of claim 45, wherein said return amount adjustment comprises a small return 
20 penalty reducing said final performance score for returns on said at least one investment less than a 

predetermined amount. 

50. The system of claim 49, wherein said small return penalty is comprised of a difference between 
an average of a short period of time return factor, an intermediate period of time return factor, and a 

25 long period of time return factor, and a penalty; wherein said penalty is equal to approximately 15% for 
upward revisions and approximately -8% for downward revisions minus a return value corresponding 
to the investment for said long period of time; and wherein each period of time return factor is a cube 
root of a return percentage over a corresponding period of time. 

30 51. The system of claim 49, wherein said small return penalty is imposed for small returns over a 
long period of time of approximately sixty days after said at least one revision. 

52. The system of claim 49, wherein said small return penalty is generated by subtracting a penalty 
from an average of a first, second and third period of time return factor, wherein for upward revisions 
35 said penalty is 0 if an average return value corresponding to the investment for a long term period of 
time is greater than 15%, else said penalty is equal to 15% minus said average return value 
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corresponding to the investment for a long term period of time; wherein for downward revisions said 
penalty is 0 if said average return value corresponding to the investment for a long term period of time 
is less than -8%, else said penalty is equal to -8% minus said average return value corresponding to the 
investment for a long term period of time; wherein said first period of time return factor is ten if an 
S average return over said first period of time is greater than ten, else said first period of time return 
factor is equal to a cube root of a product of 100 and said average return over said first period of time; 
wherein said second period of time return factor is ten if an average return over said second period of 
time is greater than ten, else said second period of time return factor is equal to a cube root of a product 
of 100 and said average return over said second period of time; and wherein said third period of time 
10 return factor is ten if an average return over said third period of time is greater than ten, else said third 
period of time return factor is equal to a cube root of a product of 100 and said average return over said 
third period of time. 

53. A system for evaluating an analyst's performance, said system comprising: 

IS means for utilizing information pertaining to at least one upward or downward revision 

including an upward or downward change in opinion of the analyst with regard to the at least one 
investment; 

means for calculating a performance score indicative of the analyst's performance relative to 
other analysts, said performance score determined at least in part by considering a measure of 
20 variability of the analyst's performance, an average historical performance of the at least one 

investment following the at least one upward or downward revision, a standard deviation of the at least 
one upward or downward revision, a number of upward or downward revisions made by the analyst, 
and a likelihood that the at least one upward or downward revision will actually produce an expected 
result; and 

25 means for comparing the analyst's performance score against performance scores of other 

analysts to produce at least one of an upward, downward or combined revision ranking. 

54. The system of claim 53, further comprising means for adjusting said performance score 
according to a modifier accentuating the number of revisions made by the analyst 

30 

55. The system of claim 53, wherein said performance score increases when a rate of return of the 
at least one investment increases and the revision is an upward revision, or when a rate of return of the 
at least one investment decreases and the revision is a downward revision, and wherein said 
performance score decreases when a rate of return of the at least one investment decreases and the 

35 revision is an upward revision, or when a rate of return of the at least one investment increases and the 
revision is a downward revision. 
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56. The system of claim S3, wherein said calculating a performance score comprises: 

determining a first average return value (ave. retumi), second average return value (ave. 
return*), and third average return value (ave. return) for the at least one revision, said first, second, and 
third average return values corresponding respectively to an average rate of return on the at least one 
revision involving the investment for a first period of time (timet), a second period of time (tim^), and 
a third period of time (time^); 

calculating, in absolute terms, a first t-stat value (t-stati) for said first average return value, a 
second t-stat value (t-statj for said second average return value, and a third t-stat value (t-stat 3 ) for said 
third average return value, said first, second, and third t-stat values calculated by utilizing said first, 
second, and third average return values, a first standard deviation (SDi), a second standard deviation 
(SD2). and a third standard deviation (SD3), a total number of revisions (revisions 0 occurring more than 
or equal to the first period of time before said evaluating, a total number of revisions (revisions^) 
occurring more than or equal to the second period of time before said evaluating, and a total number of 
revisions (revisions 3 ) occurring more than or equal to the third period of time before said evaluating; 

calculating a first degree of freedom (DFO for said first average return value, a second degree 
of freedom (DF2) for said second average return value, and a third degree of freedom (DF 3 ) for said 
third average return value; 

determining a first probability (probi) for said first period of time by utilizing said first t-stat 
value and said first degree of freedom, a second probability (proth) for said second period of time by 
utilizing said second t-stat value and said second degree of freedom, and a third probability (prob^) for 
said third period of time by utilizing said third t-stat value and said third degree of freedom; and 

generating a conditional performance score (cond. performance score) for the analyst by 
utilizing probi, proth, and prob^, which may be manipulated to produce said performance score. 



57. The system of claim 56, wherein if SD X = 0, then conditional performance score = 0 otherwise 

ave. returnl 



Tstat } 



1 - 



SDI 
^1 revisions! 



; wherein if SD2, = 0, then conditional performance score = 0 otherwise 

ave. return! 

Tstat 2 = 
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; wherein if SD3, = 0, then conditional performance score = 0 otherwise 

ave. return3 



Tstat z = 



SD3 



^revisionsi 



; wherein DF l = revisions! - 1; DF 2 = revision^ - 1; and DF 3 = revisions 3 - 1 

5 

58. The system of claim 56, wherein said probabilities are determined according to a predetermined 
look-up table using actual calculated values of: DF 2 , DF 2 , DF 3 , t-stat], t-stat 2 , and t-stat 3f unless: DFj > 
100, in which case DFj is set to 100; DF 2 > 100, in which case DF 2 is set to 100; DF 3 > 100, in which 
case DF 3 is set to 100; t-stat t > 20, in which case t-stati is set to 20; t-stat 2 > 20, in which case t-stat 2 is 

10 set to 20; t-stat 3 > 20, in which case t-stat 3 is set to 20; t-stat t < 0.01, in which case probi is set to 0; t- 
stat 2 < 0.01, in which case pro^ is set to 0; or t-stat 3 < 0.01 , in which case prot^ is set to 0. 

59. The system of claim 56, wherein 



15 wherein for upward revisions probi is subtracted from 0.5 if ave. retunii > 0, else probi is added to 0.5; 

wherein for upward revisions prot^ is subtracted from 0 JS if ave. retime > 0, else prot^ is added to 0.5; 

wherein for upward revisions prot^ is subtracted from 0.5 if ave. retum 3 > 0, else prob3 is added to 0.5; 

wherein for downward revisions probi is subtracted from 0.5 if ave. return! < 0, else probi is added to 

0.5; wherein for downward revisions probj is subtracted from 0.5 if ave. retum 2 < 0, else prot^ is added 
20 to 0.5; and wherein for downward revisions prob3 is subtracted from 0.5 if ave. return 3 < 0, else prot^ is 

added to 0.5. 

60. A computer data signal embodied in a carrier wave, said signal bearing instructions for causing 
a computer system to evaluate an analyst's performance, said instructions comprising: 
25 utilizing information pertaining to at least one revision made by the analyst involving at least 

one investment, including an upward or downward change in opinion of the analyst with regard to the 
at least one investment; 



cond _ performance _ score 



[l - (.5 + or - probi)) + [l - (5 + or - probi)] + [l - ( 5 + or - pi 



^1 Woo, 
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calculating a conditional performance score indicative of the analyst's performance relative to 
other analysts, said raw conditional performance score determined at least in part by considering a 
measure of variability of the analyst's performance, an average historical performance of the at least 
one investment following the at least one revision, a standard deviation of the at least one revision, a 
S number of revisions made by the analyst and a likelihood that the at least one revision will actually 
produce an expected result; and 

adjusting said conditional performance score according to a return amount adjustment to 
produce a final performance score. 

10 61 . The signal of claim 60, wherein the final performance score increases when a rate of return of 
the at least one investment increases and the revision is an upward revision, or when a rate of return of 
the at least one investment decreases and the revision is a downward revision, and wherein the final 
performance score decreases when a rate of return of the at least one investment decreases and the 
revision is an upward revision, or when a rate of return of the at least one investment increases and the 

IS revision is a downward revision. 



62. The signal of claim 60, wherein said return amount adjustment comprises a small return penalty 
reducing said final performance score for returns on said at least one investment less than a 
predetermined amount 

20 

63. The signal of claim 62, wherein said small return penalty is comprised of a difference between 
an average of a short period of time return factor, an intermediate period of time return factor, and a 
long period of time return factor, and a penalty; wherein said penalty is equal to approximately 15% far 
upward revisions and approximately -8% for downward revisions minus a return value corresponding 

25 to the investment for said long period of time; and wherein each period of time return factor is a cube 
root of a return percentage over a corresponding period of time. 

64. The signal of claim 62, wherein said small return penalty is generated by subtracting a penalty 
from an average of a first second and third period of time return factor, wherein for upward revisions 

30 said penalty is 0 if an average return value corresponding to the investment for a long term period of 
time is greater than 15%, else said penalty is equal to 15% minus said average return value 
corresponding to the investment for a long term period of time; wherein for downward revisions said 
penalty is 0 if said average return value corresponding to the investment for a long term period of time 
is less than -8%, else said penalty is equal to -8% minus said average return value corresponding to the 

35 investment for a long term period of time; wherein said first period of time return factor is ten if an 
average return over said first period of time is greater than ten, else said first period of time return 
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factor is equal to a cube root of a product of 100 and said average return over said first period of time; 
wherein said second period of time return factor is ten if an average return over said second period of 
time is greater than ten, else said second period of time return factor is equal to a cube root of a product 
of 100 and said average return over said second period of time; and wherein said third period of time 
S return factor is ten if an average return over said third period of time is greater than ten, else said third 
period of time return factor is equal to a cube root of a product of 100 and said average return over said 
third period of time. 

65. A computer data signal embodied in a carrier wave, said signal bearing instructions for causing 
10 a computer system to evaluate an analyst's performance, said instructions comprising: 

utilizing information pertaining to at least one upward or downward revision including an 
upward or downward change in opinion of die analyst with regard to the at least one investment; 

calculating a performance score indicative of the analyst's performance relative to other 
analysts, said performance score determined at least in part by considering a measure of variability of 
15 the analyst's performance, an average historical performance of the at least one investment following 
the at least one upward or downward revision, a standard deviation of the at least one upward or 
downward revision, a number of upward or downward revisions made by the analyst, and a likelihood 
that the at least one upward or downward revision will actually produce an expected result; and 

comparing the analyst's performance scene against performance scores of other analysts to 
20 produce at least one of an upward, downward or combined revision ranking. 

* 66. The signal of claim 65, further comprising instructions for adjusting said performance score 
according to a modifier accentuating the number of revisions made by the analyst 

25 67. The signal of claim 65, wherein said calculating a performance score comprises: 

determining a first average return value (ave. returni), second average return value (ave. 

return^, and third average return value (ave. return 3 ) for the at least one revision, said first, second, and 

third average return values corresponding respectively to an average rate of return on the at least one 

♦ 

revision involving the investment for a first period of time (timeO, a second period of time (timej, and 

30 a third period of time (times); 

calculating, in absolute terms, a first t-stat value (t-stati) for said first average return value, a 
second t-stat value (t-stat2> for said second average return value, and a third t-stat value (t-stat 3 ) for said 
third average return value, said first, second, and third t-stat values calculated by utilizing said first, 
second, and third average return values, a first standard deviation (SD0, a second standard deviation 

35 (SD2), and a third standard deviation (SD3), a total number of revisions (revisions^ occurring more than 
or equal to the first period of time before said evaluating, a total number of revisions (revisions^ 
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occurring more than or equal to the second period of time before said evaluating, and a total number of 
revisions (revisk>ns 3 ) occurring more than or equal to the third period of time before said evaluating; 

calculating a first degree of freedom (DF t ) for said first average return value, a second degree 
of freedom (DF2) for said second average return value, and a third degree of freedom (DF 3 ) for said 
5 third average return value; 

determining a first probability (probi) for said first period of time by utilizing said first t-stat 
value and said first degree of freedom, a second probability (prot>2) for said second period of time by 
utilizing said second t-stat value and said second degree of freedom, and a third probability (proba) for 
said third period of time by utilizing said third t-stat value and said third degree of freedom; and 
10 generating a conditional performance score (cond. performance score) for the analyst by 

utilizing prob lf prot>2, and prob^, which may be manipulated to produce said performance score. 

68. A computer readable medium storing instructions executable by a computer, the instructions for 
instructing the computer to effect evaluating an analyst's performance, said instructions comprising: 

15 utilizing information pertaining to at least one revision made by the analyst involving at least 

one investment, including an upward or downward change in opinion of the analyst with regard to the 
at least one investment; 

calculating a conditional performance score indicative of the analyst's performance relative to 
other analysts, said raw conditional performance score determined at least in part by considering a 

20 measure of variability of the analyst's performance, an average historical performance of the at least 
one investment following the at least one revision, a standard deviation of the at least one revision, a 
number of revisions made by the analyst, and a likelihood that the at least one revision will actually 
produce an expected result; and 

adjusting said conditional performance score according to a return amount adjustment to 

25 produce a final performance score. 

69. The computer readable medium of claim 68, wherein the final performance score increases 
when a rate of return of the at least one investment increases and the revision is an upward revision, or 
when a rate of return of the at least one investment decreases and the revision is a downward revision, 

30 and wherein the final performance score decreases when a rate of return of the at least one investment 
decreases and the revision is an upward revision, or when a rate of return of the at least one investment 
increases and the revision is a downward revision. 

70. The computer readable medium of claim 68, wherein said return amount adjustment comprises 
35 a small return penalty reducing said final performance score for returns on said at least one investment 

less than a predetermined amount. 
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71. The computer readable medium of claim 70, wherein said small return penalty is comprised of 
a difference between an average of a short period of time return factor, an intermediate period of time 
return factor, and a long period of time return factor, and a penalty; wherein said penalty is equal to 

5 approximately 15% for upward revisions and approximately -&% for downward revisions minus a 
return value corresponding to the investment for said long period of time; and wherein each period of 
time return factor is a cube root of a return percentage over a corresponding period of time. 

72. The computer readable medium of claim 70, wherein said small return penalty is generated by 
10 subtracting a penalty from an average of a first, second and third period of time return factor; wherein 

for upward revisions said penalty is 0 if an average return value corresponding to the investment for a 
long term period of time is greater than 15%, else said penalty is equal to 15% minus said average 
return value corresponding to the investment for a long term period of time; wherein for downward 
revisions said penalty is 0 if said average return value corresponding to the investment for a long term 

15 period of time is less than -8%, else said penalty is equal to -8% minus said average return value 

corresponding to the investment for a long term period of time; wherein said first period of time return 
factor is ten if an average return over said first period of time is greater than ten, else said first period of 
time return factor is equal to a cube root of a product of 100 and said average return over said first 
period of time; wherein said second period of time return factor is ten if an average return over said 

20 second period of time is greater than ten, else said second period of time return factor is equal to a cube 
root of a product of 100 and said average return over said second period of time; and wherein said third 
period of ture return factor is ten if an average return over said third period of time is greater than ten, 
else said third period of time return factor is equal to a cube root of a product of 100 and said average 
return over said third period of time. 

25 

73. A computer readable medium storing instructions executable by a computer, the instructions for 
instructing the computer to effect evaluating an analyst's performance, said instructions comprising: 

utilizing information pertaining to at least one upward or downward revision including an 
upward or downward change in opinion of the analyst with regard to the at least one investment; 

30 calculating a performance score indicative of the analyst's performance relative to other 

analysts, said performance score determined at least in part by considering a measure of variability of 
the analyst's performance, an average historical performance of the at least one investment following 
the at least one upward or downward revision, a standard deviation of the at least one upward or 
downward revision, a number of upward or downward revisions made by the analyst, and a likelihood 

35 that the at least one upward or downward revision will actually produce an expected result; and 
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comparing the analyst's performance score against performance scores of other analysts to 
produce at least one of an upward, downward or combined revision ranking. 

74. The computer readable medium of claim 73, further comprising adjusting said performance 
S score according to a modifier accentuating the number of revisions made by the analyst 

75. The computer readable medium of claim 73, wherein said calculating a performance score 
comprises: 

determining a first average return value (ave. return^, second average return value (ave. 

10 return^, and third average return value (ave. retun^) for the at least one revision, said first, second, and 
third average return values corresponding respectively to an average rate of return on the at least one 
revision involving the investment for a first period of time (timet), a second period of time (time^, and 
a third period of time (time 3 ); 

calculating, in absolute terms, a first t-stat value (t-statj for said first average return value, a 

15 second t-stat value (t-stat2) for said second average return value, and a third t-stat value (t-stat 3 ) for said 
third average return value, said first, second, and third t-stat values calculated by utilizing said first, 
second, and third average return values, a first standard deviation (SDj), a second standard deviation 
(SD2), and a third standard deviation (SD3), a total number of revisions (revisionsi) occurring more than 
or equal to the first period of time before said evaluating, a total number of revisions (revisions^ 

20 occurring more than or equal to the second period of time before said evaluating, and a total number of 
revisions (revisions3) occurring more than or equal to the third period of time before said evaluating; 

calculating a first degree of freedom (DFi) for said first average return value, a second degree 
of freedom (DF2) for said second average return value, and a third degree of freedom (DF 3 ) for said 
third average return value; 

25 determining a first probability (probi) for said first period of time by utilizing said first t-stat 

value and said first degree of freedom, a second probability (probj for said second period of time by 
utilizing said second t-stat value and said second degree of freedom, and a third probability (prot>3) for 
said third period of time by utilizing said third t-stat value and said third degree of freedom; and 
generating a conditional performance score (cond. performance score) for the analyst by 

30 utilizing probi, prob2, and prob^, which may be manipulated to produce said performance score. 

76. A method of evaluating an analyst's performance, said method comprising: 

a step for utilizing information pertaining to at least one revision made by the analyst involving 
at least one investment, including an upward or downward change in opinion of the analyst with regard 
35 to the at least one investment; 
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a step for calculating a conditional performance score indicative of the analyst's performance 
relative to other analysts, said raw conditional performance score determined at least in part by 
considering a measure of variability of the analyst's performance, an average historical performance of 
the at least one investment following the at least one revision, a standard deviation of the at least one 
5 revision, a number of revisions made by the analyst, and a likelihood that the at least one revision will 
actually produce an expected result; and 

a step for adjusting said conditional performance score according to a return amount adjustment 
to produce a final performance score. 

10 77. A method of evaluating an analyst's performance, said method comprising: 

a step for utilizing information pertaining to at least one upward or downward revision 
including an upward or downward change in opinion of the analyst with regard to the at least one 
investment; 

a step for calculating a performance score indicative of the analyst's performance relative to 
IS other analysts, said performance score determined at least in part by considering a measure of 
variability of the analyst's performance, an average historical performance of the at least one 
investment following the at least one upward or downward revision, a standard deviation of the at least 
one upward or downward revision, a number of upward or downward revisions made by the analyst, 
and a likelihood that the at least one upward or downward revision will actually produce an expected 
20 result; and 

a step for comparing the analyst's performance score against performance scores of other 
analysts to produce at least one of an upward, downward or combined revision ranking. 
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meaningful search from being carried out 



| I the description 



| [ the claims 



| | the drawings 



3. | | The failure of the nucleotide and/or amino acid sequence listing to comply with the standard provided for in Annex C of the 

Administrative Instructions prevents a meaningful search from being carried out 

[""I the written form has not been furnished or oV)es not rompry with trie stardard. 

| | the computer readable form has not been furnished or does not comply with the standard. 

4. Further comments: 



Name and maiing address of the International Searching Authority 

European Patent Office, P.B. 581 8 Patentiaan 2 

NL-2280 WRijswi* 

Tel. (+31-70) 340-2040. Tx. 31 651 epo nl, 
Fax: (+31-70) 340-3018 



Authorized officer 

Mar 'a Rodr'guez Novoa 



Form PCT/ISA/203 (July 1998) 



International Application No. PCT/US 81/11691 



FURTHER INFORMATION CONTINUED FROM PCT/ISA/ 203 



The subject-matter claimed in claims 1-28,76,77 falls under the 
provisions of Article 17(2) (a) (i) and Rule 39.1 (i. iii), PCT, such 
subject-matter relating to mathematical methods and methods of doing 
business. 

Claims 29-75 relate to a conventional system, program product, co&puter 
readable median or electrical signal for performing the business method 
of claims 1-28,76,77. Although these claims do not literally belong to 
the method category, they essentially claim protection for the same 
comnercial effect as the method claims. The International Searching 
Authority considers that searching this suhject-matter would serve no 
useful purpose. It is not at present apparent how the suhject-matter of 
the present claims may be considered defensible in any subsequent 
examination phase in front of the EPO as International Preliminary 
Examining Authority with regard to the provisions of Article 33(1) PCT 
(novelty, inventive step); see also Guidelines B-VII, 1-6). 

The applicant's attention is drawn to the fact that claims relating to 
inventions in respect of which no international search report has been 
established need not be the subject of an international preliminary 
examination (Rule 66.1(e) PCT). The applicant is advised that the EPO 
policy when acting as an International Preliminary Examining Authority is 
normally not to carry out a preliminary examination on matter which has 
not been searched. This is the case irrespective of whether or not the 
claims are amended following receipt of the search report or during any 
Chapter II procedure. If the application proceeds into the regional phase 
before the EPO, the applicant is reminded that a search may be carried 
out during examination before the EPO (see EPO Guideline C-VI, 8.5), 
should the problems which led to the Article 17(2) declaration be 
overcome. 



